SEBIS/code_trans_t5_small_commit_generation_transfer_learning_finetune

CodeTrans model is based on the t5-small model architecture . It has its own SentencePiece vocabulary model . It used transfer-learning pre-training on 7 unsupervised datasets in the software development domain . It is then fine-tuned on the git commit message generation task for the java commit changes . This model is trained on tokenized git commit: it works best with tokenized commit . It can be used on unparsed and untokenized commit changes, however, if the change is tokenized, the performance should be better. It has a total of approximately 220M parameters and was trained using the encoder-decoder architecture. The optimizer used is,