tartuNLP/EstBERT

The EstBERT model is a pretrained BERTBase model exclusively trained on Estonian cased corpus on both 128 and 512 sequence length of data . The model performs better in parts of speech (POS), name entity recognition (NER), rubric, and sentiment classification tasks compared to mBERT and XLM-RoBERTa . For training the model we used the Estonian National Corpus 2017, which was the largest Estonian language corpus available at the time . The comparative results can be found below;. The results can also be downloaded from here, the pretrained model is used to train the model and the model is trained on 128 or 512 sequences of data. The model transformer library,