monsoon-nlp/hindi-bert

This is a first attempt at a Hindi language model trained with Google Research's ELECTRA . The small model has comparable results to Multilingual BERT on BBC Hindi news classification and on Hindi movie reviews / sentiment analysis (using SimpleTransformers) A larger model (Hindi-TPU-Electra) using ELECTRA base size outperforms both models using Electra base size, but does not perform as well on the BBC news classification task . The corpus is two files: Hindi CommonCrawl deduped by OSCAR and Hindi Wikipedia (https://dumps.wikimedia.org/hiwiki/ ) + WikiExtractor to txt/transformers on txt .,