krevas/finance-koelectra-small-discriminator

ELECTRA is a new method for self-supervised language representation learning . It can be used to train transformer networks using relatively little compute . The current version of the model is trained on a financial news data of Naver news . The final training corpus has a size of 25GB and 2.3B tokens . This model was trained a cased model on a TITAN RTX for 500k steps . All models are available on the Huggingface model hub, which is open to the public on GitHub . For more information about ELECTRA can be found in the ICLR paper or in the official ELECTRA repository on GitHub. For more details about the ELECTRA paper, visit the I,