google/t5-xxl-ssm-wq

The model was fine-tuned on 100% of the train splits of Web Questions (WQ) for 10k steps . The model can be used as follows for closed book question answering: t5_qa_model.from_pretrained("google/t5-xxl-ssm-wq") The model is pre-trained using T5's denoising objective on C4, subsequently additionally pre- trained using REALM's salient span masking objective on Wikipedia. The model performs competitively with systems that explicitly retrieve answers from an external knowledge source when answering questions. To facilitate reproducibility and future work, we release our code and trained models at https://,