mrm8488/longformer-base-4096-finetuned-squadv2

The Longformer-base-4096 model fine-tuned on SQuAD v2 for Q&A downstream task . It supports sequences of length up to 4,096 . It uses a combination of a sliding window (local) attention and global attention . Global attention is user-configured based on the task to allow the model to learn task-specific representations . The training script is a slightly modified version of this one . Check out more about this dataset and others in NLP Viewer for more information about the training script and other models in the NLP toolkit . For example, here is a training script for the training model in action: The model is a BERT,