SEBIS/legal_t5_small_trans_de_en

The legal_t5_small_trans_de_en model was trained on three parallel corpus from jrc-acquis, europarl and dcep . This model could be used for translation of legal texts from Deustch to English . It has about 60 million parameters and was trained using a single TPU Pod V3-8 for 250K steps in total, using sequence length 512 (batch size 4096) The optimizer used is AdaFactor with inverse square root learning rate schedule for pre-training . When the model is used for a test, the model achieves the following results:. The model is based on the t5-small model, which scales the baseline,