avichr/heBERT

HeBERT is a Hebrew pretrained language model based on Google's BERT architecture and it is BERT-Base config (Devlin et al. 2018) We evaluated the model on emotion recognition and sentiment analysis, for a downstream task . We are still working on our model and will edit this page as we progress .Note that we have released only sentiment analysis (polarity) at this point, emotion detection will be released later on . The model was trained on three datasets: a Hebrew version of OSCAR (Ortiz, 2019): 9.8 GB of data, including 1 billion words and over 20.8 millions sentences . A Hebrew dump of Wikipedia: ~650,