lordtt13/COVID-SciBERT

SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks . The expansion is done using the papers present in the COVID-19 Open Research Dataset Challenge (CORD-19) Only the abstracts have been used and vocabulary was pruned and added to the existing scivocab to best match the training corpus . There is a growing urgency for these approaches because of the rapid acceleration in new coronavirus literature, making it difficult for the medical research community to keep up with the rapid pace of the new literature . The training script is present here . There are actually two datasets that have been,