dbmdz/electra-small-turkish-cased-discriminator

ELECTRA is a new method for self-supervised language representation learning . It can be used to pre-train transformer networks using relatively little compute . The final training corpus has a size of 35GB and 44,04,976,662 tokens . The ELECTRA model is trained on a filtered and sentence segmented version of the Turkish OSCAR corpus, a recent Wikipedia dump, various OPUS corpora and a special corpus provided by Kemal Oflazer . It is possible to download both cased and uncased models from the Hugging Face team's S3 storage . The ElectRA small cased model can be loaded like: AutoModelWithLMHead, AutoTokenizer,