microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext

PubMedBERT (abstracts + full text) is pretrained from scratch using abstracts from PubMed and full-text articles from PubMedCentral . This model achieves state-of-the-art performance on many biomedical NLP tasks, and currently holds the top score on the Biomedical Language Understanding and Reasoning Benchmark . Recent work shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models . If you find this paper useful in your research, please cite it in the following paper: "Pretraining for Biomedical Natural Language Processing," or ",