albert-xlarge-v1

ALBERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . It was introduced inthis paper and first released in this repository . Version 2 is different from version 1 due to different dropout rates, additional training data, and longer training . This model is uncased: it does not make a difference between English and English . It's mostly intended to be fine-tuned on downstream tasks that use the whole sentence (potentially masked)to make decisions, such as sequence classification, token classification or question answering . For tasks such as text-generation you should look at model like GPT2, like that of GPT1 .,