madlag/bert-base-uncased-squadv1-x1.84-f88.7-d36-hybrid-filled-v1

The model was fine-tuned from the HuggingFace BERT base uncased checkpoint on SQuAD1.1 and distilled from the equivalent model csarron/bert-base-uncased-squad-v1 . In terms of accuracy, its F1 is 88.72, compared with 88.5 for BERT-base, a F1 gain of 0.22.22 . The model contains 36.0% of the original weights overall (remaining weights are mostly in the input/ouput embeddings). With a simple resizing of the linear matrices it ran 1.84x as fast as BERT on the evaluation. This is possible,