indigo-ai/BERTino

The repository hosts BERTino, an Italian DistilBERT model pre-trained byindigo.ai.ai on a large general-domain Italian corpus . The final corpus counts 14 millions of sentences for a total of 12 GB of text . The pre-training corpus that we used is the union of the.otypesPaisa and.ItWaCcorpora. The final . corpus is a union of. the union . of the two.otypes that are pre--trained and can be  fine-tuned for every downstream task . We evaluated Bertino on the . Italian ParTUT, Italian ISDT, French WikiNER and multi-class sentence classification tasks,