albert-base-v2

ALBERT Base v2 is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . The model is uncased: it does not make a difference between English and English . It has the same computational cost as a BERT-like architecture with the same number of hidden layers as it has to iterate through (repeating) layers . The second version of the base model is different from version 1 due to different dropout rates, additional training data, and longer training . This model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked)to make decisions, such as sequence classification, token classification,