google/t5-11b-ssm-nq

The model was fine-tuned on 100% of the train splits of Natural Questions (NQ) for 10k steps . The model can be used as follows for closed book question answering: t5_qa_model = AutoModelForSeq2SeqLM, AutoTokenizer.from_pretrained("google/t5-11b-ssm-nq") The model is pre-trained using T5's denoising objective on C4, subsequently additionally pre- trained using REALM's salient span, and finally fine-trained on Natural Questions . We show that this approach scales with model size and performs competitively with open-domain systems that explicitly retrieve,