albert-base-v1

ALBERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . The model is uncased: it does not make a difference between English and English . It has the same computational cost as a BERT-like architecture with the same number of hidden layers as it has to iterate through (repeating) layers . It shares its layers across its Transformer, all layers have the same weights . This model is the first version of the base model, and version 2 is different from version 1 due to different dropout rates, additional training data, and longer training . For tasks such as text-generation you should look at model like G,