mrm8488/roberta-base-1B-1-finetuned-squadv2

Machine Learning for Language pretrained RoBERTa on smaller datasets (1M, 10M, 100M, 1B tokens) They released 3 models with lowest perplexities for each pretraining data size out of 25 runs . The data reproduces that of BERT: They combine English Wikipedia and a reproduction of BookCorpus using texts from smashwords in a ratio of approximately 3:1 . The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: python transformers/examples/question-answering/run_squad.py . To do well on SQuAD2.0, systems must not only answer questions when possible,,