NlpHUST/vi-electra-base

ELECTRA is a method for self-supervised language representation learning . It can be used to pre-train transformer networks using relatively little compute . At small scale, ELECTRA achieves strong results even when trained on a single GPU . At large scale, it achieves state-of-the-art results on the SQuAD 2.0 dataset . The ELECTRA models are trained to distinguish "real" input tokens vs . "fake" input . tokens generated by another neural network, similar to the discriminator of a . GAN . The project is based on the work of Nha Nguyen Van (nha282@gmail.com). For more information about the project, please contact N,