mrm8488/GPT-2-finetuned-common_gen

GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion . It was trained to guess the next word in sentences . The model is best at what it was pretrained for however, which is generating texts from a sequence of continuous text of a certain length and the targets are the same sequence, shifting one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token i only uses the inputs from 1 to 1 but not the future tokens . This way, the model learns an inner representation of the English language that can then be used to,