airesearch/wangchanberta-base-wiki-syllable

The architecture of the pretrained model is based on RoBERTa [Liu et al., 2019]. The model was trained on the Wikipedia dump articles on 20 August 2020 (dumps.wikimedia.org/thwiki/20200820/). We opt out lists, tables, and tables . We use a Thai syllable-level dictionary-based tokenizer denoted as syllable from PyThaiNLP [Phatthiyaphaibun et al. 2016]. The total number of word-level tokens in the vocabulary is 59,235 . We sample sentences contigously to have the length of at most 512 tokens . For some sentences that overlap the boundary of,