facebook/rag-token-base

This is a non-finetuned version of the RAG-Token model of the paper Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks . The question encoder can be any model that can be loaded with AutoModelForSeq2SeqLM . The retriever should be a RagRetriever instance . The model is uncased so that all capital input letters are converted to lower-case . It uses the dummy retriever as a default, but better results are obtained by using the full retriever . It can be fine-tuned as follows: The model can be used to fine-tune as well as to train on loss .,