valhalla/distilbart-mnli-12-1

DistilBart-MNLI is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here . We just copy alternating layers from the data and finetune more on the same data . This is a very simple and effective technique, as we can see the performance drop is very little .Detailed performace trade-offs will be posted in this sheet . You can find the logs of these trained models in this wandb project, this is the distillbart-nli project . If you want to train these models yourself, clone the distilbart-mnl repo and follow the steps below .,