cl-tohoku/bert-large-japanese-char

This is a BERT model pretrained on texts in the Japanese language . It processes input texts with word-level tokenization based on the Unidic 2.1.2 dictionary . The model architecture is the same as the original BERT large model; 24 layers, 1024 dimensions of hidden states, and 16 attention heads . It is trained with whole word masking enabled for the masked language modeling (MLM) objective . The models are trained on the Japanese version of Wikipedia . They are distributed under the terms of the Creative Commons Creative Commons Attribution-ShareAlike 3.0.0-3.0 . The training took about 5 days to finish. The training of each model is,