aodiniz/bert_uncased_L-2_H-512_A-8_cord19-200616

Bert L-2 H-512 fine-tuned on MLM (CORD-19 2020/06/16) with 2 Transformer layers and hidden embedding of size 512 . The model was referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models, on the importance of pre-training compact models . The training model is based on the CORD19-200616-dataset (as released on 2020/07/16). Training the model with the model is python python run_language_modeling.py.py: --train_data_file {cord19-0616- dataset} --train,