hfl/chinese-electra-base-generator

Google and Stanford University released a new pre-trained model called ELECTRA . It has a much compact model size and relatively competitive performance compared to BERT and its variants . Chinese ELECTRA models based on the official code of ELECTRA could reach similar or even higher scores on several NLP tasks with only 1/10 parameters . The Joint Laboratory of HIT and iFLYTEK Research (HFL) has released the Chinese ElectRA models . [Please use Electra forPreTraining for discriminator and ElectraForMaskedLM for generator if you are re-training these models . The Chinese Electra models could be used to test new models in Chinese BERT series or XLNet .,