aodiniz/bert_uncased_L-4_H-256_A-4_cord19-200616

Bert L-4 H-256 fine-tuned on MLM (CORD-19 2020/06/16) with 4 Transformer layers and hidden embedding of size 256 . The model was referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models, on the importance of pre-training compact models, in an article published by MIT and Google . The Bert model has been fine-tuned for MLM on CORD19-200616-dataset (as released on 2020/07/16). The model is based on the data set of the Cord19-0616-Dataset, as released on,