nlpaueb/legal-bert-base-uncased

LEGAL-BERT is a family of BERT models for the legal domain . It is intended to assist legal NLP research, computational law, and legal technology applications . A light-weight model pre-trained from scratch on legal data with competitive perfomance is also available . We trained BERT using the official code provided in Google BERT's GitHub repository (https://github.com/google-research/bert) The training set-up: 1 million training steps with batches of 256 sequences of length 512 with an initial learning rate 1e-4 . The pre-training corpora include:116,062 documents of EU legislation, 19,867 cases from European Court,