google/t5-large-ssm

The model should be fine-tuned on a question answering downstream task before it is useable for closed book question answering . We show that this approach scales with model size and performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source when answering questions . To facilitate reproducibility and future work, we release our code and trained models at https://goo.gle/t5-cbqa. The model was pre-trained using T5's denoising objective on C4 and subsequently additionally pre- trained using REALM's salient span masking objective on Wikipedia . The model is now being used by Google's T5 for Closed Book Question Answering,