datificate/gpt2-small-spanish

GPT2-small-spanish is a state-of-the-art language model for Spanish based on the GPT-2 small model . It was trained on Spanish Wikipedia using Transfer Learning and Fine-tuning techniques . The training took around 70 hours with four GPU NVIDIA GTX 1080-Ti with 11GB of DDR5 and around 3GB of (processed) training data . The model was fine-tuned from the English pre-trained model using the Hugging Face libraries (Transformers and Tokenizers) wrapped into the fastai v2 Deep Learning framework . The data used for this model come from Spanish Wikipedia, which contains a lot of unfiltered content from the,