cl-tohoku/bert-base-japanese

This is a BERT model pretrained on texts in the Japanese language . The model processes input texts with word-level tokenization based on the IPA dictionary, followed by WordPiece subword tokenization . Text files used for the training are 2.6GB in size, consisting of approximately 17M sentences. The model is trained on Japanese Wikipedia as of September 1, 2019. The code for the pretraining is available at cl-tohoku/bert-japanese.com . The training corpus is used to extract plain texts from a dump file of Wikipedia articles. The vocabulary size is 32000. The training data is 32,000 words. The models are distributed under the,