gsarti/covidbert-nli

CovidBERT was trained by DeepSet on AllenAI's CORD19 Dataset of scientific articles about coronaviruses . The model uses the original BERT wordpiece vocabulary and was fine-tuned on SNLI and MultiNLI datasets using the sentence-transformers library to produce universal sentence embeddings [1] using the average pooling strategy and a softmax loss . The performance was evaluated on the test portion of the STS dataset using Spearman rank correlation and compared to the performances of similar models obtained with the same procedure to verify its performances . An example usage for similarity-based scientific paper retrieval is provided in the Covid-19 Semantic Browser repository,