pierreguillou/gpt2-small-portuguese

GPorTuguese-2 is a state-of-the-art language model for Portuguese based on the GPT-2 small model . It was trained on Portuguese Wikipedia using Transfer Learning and Fine-tuning techniques in just over a day, on one GPU NVIDIA V100 32GB and with a little more than 1GB of training data . The model is now available on Hugging Face . For further information or requests, please go to "Faster than training from scratch — Fine-tuneing the English Gpt-2 in any language with Hugging face and fastai v2 (practical case with Portuguese)" The model uses internally a mask-mechanism,