google/t5-xxl-ssm-wqo

The model was fine-tuned on 90% of the train splits of Web Questions (WQ) for 20k steps . It was pre-trained using T5's denoising objective on C4 . The model can be used as follows for closed book question answering: t5_qa_model = AutoModelForSeq2SeqLM, AutoTokenizer. (T5-11b-ssm-wqo) The model performs competitively with open-domain systems that explicitly retrieve answers from an external knowledge source . To facilitate reproducibility and future work, we release our code and trained models at https://goo.gle/t5-cb,