akhooli/gpt2-small-arabic-poetry

The GPT2-small-arabic (trained on Arabic Wikipedia) and this model have several limitations in terms of coverage and training performance . Use them as demonstrations or proof of concepts but not as production code . This pretrained model used the Arabic Poetry dataset from 9 different eras with a total of around 40k poems . The dataset was trained using Simple Transformers library on Kaggle, using free GPU . The model was trained (fine-tuned) based on the gpt2-Small-ar-abic transformer model . The data was used to evaluate the final perplexity reached ws 76.3, loss: 4.33. The model has several limitations .,