dbddv01/gpt2-french-small

A small french language model for french text generation (and possibly more NLP tasks...) The model is based on openai GPT-2 small model . It was trained on a very small (190Mb) dataset from french Wikipedia using Transfer Learning and Fine-tuning techniques in just over a day, on one Colab pro with 1GPU 16GB . It is a proof-of-concept that makes possible to get a language model in any language with low ressources . It speaks french with some minor grammar issues, coherence of text is somehow limited . The model can be improved by using larger dataset under larger powerful training infrastructure. At least this one can be used for,