distilbert-base-multilingual-cased

DistilBERT is a distilled version of the BERT base multilingual model . The model is trained on the concatenation of Wikipedia in 104 languages listed here . It has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base) The code for the distillation process can be found here: "DistilberT: smaller, faster, cheaper and lighter" On average DistilmBERT (computed) is twice as fast as mBERt-base . It is cased: it does make a difference between English and English. The model has six layers and 768 dimensions . It does make,