squeezebert/squeezebert-mnli-headless

SqueezeBERT has been pretrained for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective and finetuned on the Multi-Genre Natural Language Inference (MNLI) dataset . This is a "headless" model with the final classification layer removed, and this will allow Transformers to reinitialize the final layer before you begin finetuning on your data . The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with grouped convolutions . The authors found that SqueezBERT is 4.3x faster than bert-base-uncased on,