vinai/bertweet-base

BERTweet is the first large-scale language model pre-trained for English Tweets . It is trained based on the RoBERTa pre-training procedure, using the same model configuration as BERT-base . The corpus used to pre-train Bertweet consists of 850M EnglishTweets (16B word tokens ~ 80GB) and 845M Tweets streamed from 01/2012 to 08/2019 . The general architecture and experimental results can be found in our paper: "BERTweet: A pre-trained language model for English tweets," "Part-of-speech tagging," "Named entity recognition" and "Text classification" are examples of the BERT,