facebook/rag-sequence-base

This is a non-finetuned version of the RAG-Sequence model of the paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks . The question encoder can be any model that can be loaded with AutoModelForSeq2SeqLM . The retriever should be a RagRetriever instance . The model is uncased so that all capital input letters are converted to lower-case . It can be fine-tuned as follows:. The model uses the dummy retriever as a default. Better results are obtained by using the full retriever,  by setting config.index_name="legacy" and config.,