nreimers/BERT-Small-L-4_H-512_A-8

This is a port of the BERT-Small model to Pytorch . It uses 4 layers, a hidden size of 512 and 8 attention heads . It is based on the size of the original version of BERT Small . It has 4 layers and 4 attention heads with 512 and a 512-A-8 head size . It's a port for the original BERT small-scale version of the model that was created by BERTSmall . It was developed in the 1990s and is now being used by PyTorch for the first time in the wild . The original version has been released on December 31, 2010 . It will be available in the next version of this article.