indolem/indobert-base-uncased

IndoBERT is the Indonesian version of BERT model . We train the model using over 220M words, aggregated from three main sources . We trained the model for 2.4M steps (180 epochs) with the final perplexity over the development set being 3.97 (similar to English BERT-base) IndoLEM - an Indonesian benchmark that comprises of seven tasks for the Indonesian language, spanning morpho-syntax, semantics, and discourse . The paper is published at the 28th COLING 2020. Please refer to the benchmarks for more details about the benchmarks. If you use our work, please cite the work, or use the hashtag #openergasm,