akhooli/gpt2-small-arabic

GPT2-Small-Arabic (trained on Arabic Wikipedia) has several limitations in terms of coverage (Arabic Wikipeedia quality, no diacritics) and training performance . The training was done using Fastai2 library on Kaggle, using free GPU . The final perplexity reached was 72.19, loss: 4.28, accuracy: 0.307. Use as demonstration or proof of concepts but not as production code . The model was trained on the Arabic Wikipedia dump (around 900 MB) and trained using the model's training procedure . The data dump was around 900 MB and the model was used to generate text and poetry (fine-tuned model,