Ochiroo/tiny_mn_gpt

GPT-2 is a transformers model pretrained on a very small corpus of Mongolian news data in a self-supervised fashion . This means it was trained to guess the next word in sentences . Trained on 500MB of news dataset (IKON) on . RTX 2060 . It can use lots of publicly available data, with an automatic process to generate inputs and labels from those texts . The model was trained on the raw texts only, with no humans labelling them in any way (which is why it can use public . available data) Trained to guess what word in each word in a series of sentences and to predict the next . word in the series of words,