squeezebert/squeezebert-mnli

SqueezeBERT has been pretrained for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective and finetuned on the Multi-Genre Natural Language Inference (MNLI) dataset . This model is case-insensitive and is 4.3x faster than bert-base-uncased on a Google Pixel 3 smartphone . The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with grouped convolutions . Finetuning with distillation is not yet implemented in this repo . If the author (Forrest Iandola - forrest.dnn@gmail.,