dbmdz/electra-base-turkish-cased-discriminator

ELECTRA is a new method for self-supervised language representation learning . It can be used to pre-train transformer networks using relatively little compute . ELECTRA models are trained to distinguish "real" input tokens vs "fake" input . The final training corpus has a size of 35GB and 44,04,976,662 tokens . The ELECTRA model is trained on a filtered and sentence segmented version of the Turkish OSCAR corpus, a recent Wikipedia dump, various OPUS corpora and a special corpus provided by Kemal Oflazer . It is possible to download both cased and uncased models from the Hugging Face team's S3 storage . All models are,