SEBIS/code_trans_t5_base_commit_generation_multitask

This model is trained on tokenized git commit: it works best with tokenized commit . It can be used on unparsed and untokenized commit changes . The model could be used to generate git commit message for the git commit changes or be fine-tuned on other relevant tasks . It has a total of approximately 220M parameters and was trained using the encoder-decoder architecture. The optimizer used is AdaFactor with inverse square root learning rate schedule for pre-training. The model was trained on a single TPU Pod V3-8 for 480,000 steps in total, using sequence length 512 (batch size 4096) The model has its own SentencePiece,