jcblaise/gpt2-tagalog

This is a prototype GPT-2 model of the smallest variant, trained using a combination of WikiText-TL-39 and the NewsPH-Raw datasets . The model was trained with two language modeling datasets for Tagalog . The weights are provided in both PyTorch and TensorFlow and can be used with ease via the HuggingFace Transformers library . We discourage use of the model in systems and use-cases where the generated output is required to be true . As this model is currently a prototype, bias was not thoroughly studied . We plan to train larger models with larger corpora in the future . This model is part of a much larger work-in-progress, and as,