ethanyt/guwenbert-base

GuwenBERT is a RoBERTa model pre-trained on Classical Chinese . It can be fine-tuned for downstream tasks, such as sentence breaking, punctuation, named entity recognition, and so on . The training data is daizhige dataset which is contains of 15,694 books in Classical Chinese, covering Buddhism, Confucianism, Medicine, History, Zi, Yi, Shizang, Taoism, and Jizang . The total number of characters is 1.7B (1,743,337,673) The vocabulary is constructed from this data set and the size is 23,292 . The models are trained on 4 V100 GPUs for,