google/bert_uncased_L-2_H-512_A-8

The 24 BERT miniatures are referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models (English only, uncased, trained with WordPiece masking). The smaller BERT models are intended for environments with restricted computational resources . They are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher . The BERT-Base model in this release is included for completeness only; it was re-trained under the same regime as the original model . You can download the 24 models either from the official BERT Github page, or via HuggingFace from the links below .,