Darkrider/covidbert_medmarco

CovidBERT trained by DeepSet on AllenAI's CORD19 Dataset of scientific articles about coronaviruses . The model uses the original BERT wordpiece vocabulary and was subsequently fine-tuned on the SNLI and the MultiNLI datasets using the sentence-transformers library to produce universal sentence embeddings [1] using the average pooling strategy and a softmax loss . It is further fine tuned on the Med-Marco Datasets . The beauty of MedSyn is that the terms are more general human conversation lingo and not terms based on scientific literature . MacAvaney et.al in their paper titled “SLEDGE-Z:,