SEBIS/legal_t5_small_cls_en

The legal_t5_small_cls_en model is based on the t5-small model and was trained on a large corpus of parallel text . It scales the baseline model of t5 down by using dmodel = 512, dff = 2,048, 8-headed attention, and only 6 layers each in the encoder and decoder. This variant has about 60 million parameters. The model could be used for classification of legal texts written in English. It was first released in the PyTorch repository. It has been trained on JRC-ACQUIS dataset consisting of 19 Thousand texts. The optimizer used is AdaFactor with inverse square root learning rate schedule,