toastynews/xlnet-hongkongese-base

XLNet trained exclusively with data from Hong Kong . A signaficant amount of Hongkongese/Cantonese/Yue is included in the training data . It may offer better performance for tasks catering to the langauge usage of Hong Kongers . The training data consists of mostly news articles and blogs . Yue Wikipedia is used which is much smaller than Chinese Wikipedia; this model will lack the breath of knowledge compared to other Chinese models . Further finetuning will be needed for use on downstream tasks . There is probably a bias towards formal language usage. For text generation, a longer context will help generate better text. Overall result is not as good as GPT-2,