albert-large-v1

ALBERT Large v1 is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . It does not make a difference between English and English . The model has the following configuration: .24 repeating layers with the same number of (repeating) layers. Therefore, all layers have the same weights. Using repeating layers results in a small memory footprint, however, the computational cost remains similar to a BERT-like architecture . This model has been developed by the Hugging Face team . It is the first version of the large model. Version 2 is different from version 1 due to different dropout rates, additional training data, and longer training .,