aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616

Bert L-10 H-512 fine-tuned on MLM (CORD-19 2020/06/16) with 10 Transformer layers and hidden embedding of size 512 . The model was referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models, on the importance of pre-training compact models . The training model is based on the Cord19-200616-dataset (as released on 2020/07/16).Training the model with the model is python python run_language_modeling.py.py: --train_data_file {cord19/0616- dataset} --train,