hfl/chinese-electra-180g-small-ex-discriminator

Google and Stanford University released a new pre-trained model called ELECTRA, which has a much compact model size and relatively competitive performance compared to BERT and its variants . The Joint Laboratory of HIT and iFLYTEK Research (HFL) has released the Chinese ELECTRA models based on the official code of ELECTRA . ELECTRA-small could reach similar or even higher scores on several NLP tasks with only 1/10 parameters compared to . BERT . The new model is trained on 180G data, we recommend using this one than the original version . The ELECTRA project is based on ELECTRA: https://://github.com/google-research/electra,