amoux/roberta-cord19-1M7k

The model is based on RoBERTa and was pre-trained on 1.7 million sentences . The training corpus was taken from papers taken from Semantic Scholar's CORD-19 historical releases . I used the full-text "body_text" of the papers in training (details below). The model was trained on 1,687,124 sentences and was used for evaluation . The model has been dubbed 'roberta-cord19-1M7k' and 'robera-CORD19-2M7K' The training was based on the data from the CORD19 Open Research Dataset's COD-19 open research paper, published in,