Cinnamon/electra-small-japanese-generator

We provide a Japanese ELECTRA-Small model, as described in ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators . Our pretraining process employs subword units derived from the Japanese Wikipedia, using the Byte-Pair Encoding method and building on an initial tokenization with mecab-ipadic-NEologd . For optimal performance, please take care to set your MeCab dictionary appropriately . For example, ElectraForMaskedLM was created using the BertJapaneseTokenizer from the model of Electra for MaskedLM. It is based on the model used by the model for the model and the definition of the model itself .,