monsoon-nlp/hindi-tpu-electra

Hindi language model outperforms Multilingual BERT on Hindi movie reviews / sentiment analysis (using SimpleTransformers) You can get higher accuracy using ktrain + TensorFlow, where you can adjust learning rate and other hyperparameters . The corpus is two files: Hindi CommonCrawl deduped by OSCAR and WikiExtractor to txt . The model is created with HuggingFace Tokenizers; you can increase vocabulary size and re-train; remember to change ELECTRA vocab_size . A smaller model (Hindi-BERT) performs better on a BBC news classification task . The Hindi-language model is trained with ELECTRA base size settings and training Co,