google/muril-base-cased

MuRIL: Multilingual Representations for Indian Languages is a BERT model pre-trained on 17 Indian languages and their transliterated counterparts . We have released the model (with the MLM layer intact, enabling masked word predictions) in this repository . We also released the encoder on TFHub with an additional pre-processing module, that processes raw text into the expected input format for the model . We use a training paradigm similar to multilingual bert, with a few modifications as listed: translation and transliteration segment pairs in training as well . We keep an exponent value of 0.3 and not 0.7 for upsampling, shown to enhance low-resource,