albert-xlarge-v2

ALBERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . It was introduced inthis paper and first released in this repository . Version 2 is different from version 1 due to different dropout rates, additional training data, and longer training . It has better results in nearly all downstream tasks . For tasks such as text-generation you should look at model like GPT2. It's mostly intended to be fine-tuned on a downstream task, such as sequence classification, token classification or question answering . You can use the raw model for either masked language modeling or next sentence prediction, but you can use this model directly with a pipeline,