Cinnamon/electra-small-japanese-discriminator

We provide a Japanese ELECTRA-Small model, as described in ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators . Our pretraining process employs subword units derived from the Japanese Wikipedia, using the Byte-Pair Encoding method and building on an initial tokenization with mecab-ipadic-NEologd . For optimal performance, please take care to set your MeCab dictionary appropriately . How to use the discriminator in transformers: How to train text encoders with a discriminator .How to train transformers? Use the discriminators to learn more about how to train your language with your language. How to test your,