Wikidepia/IndoT5-base

IndoT5 only pretrained on denoising task, fine-tuning is still needed . It's available in three sizes (small, base and large) Trained 500k steps . Using script provided by unicamp-dl on PTT5 Repository. Trained on various corpus. The model is pretrained using OSCAR Corpus (OpenSubtitle, Tanzil, etc) and OpenSubtitle (OPUS) Corpus. The training data is trained using pre-trained versions of OSCAR, Open Subtitle, TensorFlow and TPU (TensorFlow Research Cloud) For more information, visit http://www.tensorflowresearch.com/tensorFlow,