cahya/gpt2-small-indonesian-522M

It is GPT2-small model pre-trained with indonesian Wikipedia using a causal language modeling (CLM) objective . This model is uncased: it does not make a difference between indonesia and Indonesia . You can use this model directly with a pipeline for text generation . The inputs are sequences of 128 consecutive tokens . The texts are tokenized using a byte-level version of ByteBoding (BPE) Pair Encoding). The input text is a text with a unicode vocabulary size of 52,000 characters and a vocabulary of 52 characters . The model was pre- trained with 522MB of Wikipedia data with a .elements of a . byte-,