madlag/bert-base-uncased-squadv1-x1.16-f88.1-d8-unstruct-v1

The model was fine-tuned from the HuggingFace BERT base uncased checkpoint on SQuAD1.1, and distilled from the model bert-large-uncased-whole-word-masking-finetuned-squad . In terms of accuracy, its F1 is 88.11, compared with 88.5 for BERT-base, a F1 drop of 0.39 . The model contains 28.0% of the original weights overall (the embeddings account for a significant part of the model, and they are not pruned by this method). With a simple resizing of the linear matrices it ran 1.16x,