lewtun/distilbert-base-uncased-distilled-squad-v1

This approach replicates the DistilBERT (D)" model from Table 2 of the DistillBERT paper . In this approach, a DistilberT student is fine-tuned on SQuAD v1.1, but with a BERT model acting as a teacher for a second step of task-specific distillation . The model was trained on the SQuad v1 dataset which can be obtained from the datasets library as follows: import load_dataset into datasets . The scores were calculated using the squad metric from datasets from datasets. They were used to calculate the scores using the metric from a squad metric . The training procedure and evaluation results were evaluated using the,