madlag/bert-base-uncased-squadv1-x2.44-f87.7-d26-hybrid-filled-v1

The model was fine-tuned from the HuggingFace BERT base uncased checkpoint on SQuAD1.1, and distilled from the equivalent model csarron/bert-base-uncased-squad-v1 . In terms of accuracy, its F1 is 87.71, compared with 88.5 for BERT-base, a F1 drop of -0.79 . The model contains 42.0% of the original weights overall (the embeddings account for a significant part of the model, and they are not pruned by this method). With a simple resizing of the linear matrices it ran 2.44x as fast as B,