allenai/led-large-16384

This notebook shows how led-large-16384 can effectively be fine-tuned on a downstream task . The model is especially interesting for long-range summarization and question answering . This notebook is based on Allenai's Longformer Encoder-Decoder (LED) and Bart-large's position embedding matrix was copied 16 times to process 16K tokens . The notebook is published by Allenai at http://://://www.ancient.com/led-large/16384//. The notebook was written by Iz Beltagy, Matthew E. Peters, Arman Cohan and Arman P. Cohan on August 31, 2013, and is published on August 28,,