cl-tohoku/bert-base-japanese-char-v2

This is a BERT model pretrained on texts in the Japanese language . It processes input texts with word-level tokenization based on the Unidic 2.1.2 dictionary . The model architecture is the same as the original BERT base model; 12 layers, 768 dimensions of hidden states, and 12 attention heads . It is trained with whole word masking enabled for the masked language modeling (MLM) objective . The models are trained on the Japanese version of Wikipedia . They are distributed under the terms of the Creative Commons Creative Commons Attribution-ShareAlike 3.0.0-3.0 . The training took about 5 days to finish. The training of each model is,