vinai/phobert-base

PhoBERT pre-trained models are the state-of-the-art language models for Vietnamese (Pho, i.e. "Phở", is a popular food in Vietnam): Two PhoBerT versions of "base" and "large" are the first public large-scale monolingual language models pre-training for Vietnamese . PhoberT is based on RoBERTa, which optimizes the BERT pre- training procedure for more robust performance . The general architecture and experimental results can be found in our EMNLP-2020 Findings paper: "Phobert: Pre-trained languages for Vietnamese," "phobert," ",