cl-tohoku/bert-base-japanese-char-whole-word-masking

This is a BERT model pretrained on texts in the Japanese language . The model processes input texts with word-level tokenization based on the IPA dictionary . It is trained with the whole word masking enabled for the masked language modeling (MLM) objective . The training corpus is 2.6GB in size, consisting of approximately 17M sentences . The models are distributed under the terms of the Creative Commons Creative Commons Attribution-ShareAlike 3.0 license . The codes for the pretraining are available at cl-tohoku/bert-japanese. The model architecture is the same as the original BERT base model; 12 layers, 768 dimensions of hidden states, and,