valhalla/longformer-base-4096-finetuned-squadv1

This is longformer-base-4096 model fine-tuned on SQuAD v1 dataset for question answering task . The model was trained on google colab v100 GPU . It is a BERT-like model for long documents . The input sequence must have three sep tokens, i.e the sequence should be encoded like this. If you encode the question and answer as a input pair, then the tokenizer already takes care of that, you shouldn't worry about it. The model can handle sequences with upto 4096 tokens. It isn't yet supported in pipeline . I'll update this card once the support has been added to the project . The LongformerForQuestion,