allegro/herbert-base-cased

HerBERT is a BERT-based Language Model trained on Polish Corporaverageusing MLM and SSO objectives with dynamic masking of whole words . The training dataset was tokenized into subwords using CharBPETokenizer a character level byte-pair encoding with a vocabulary size of 50k tokens . The tokenizer itself was trained with a tokenizers library . The language model was trained by Machine Learning Research Team at Institute of Computer Science, Polish Academy of Sciences . The model has been trained using transformers in version 2.9.9 and is available to download from version 4.0.0 or from version 5.1.0 for the latest version of HerbertTokenizer,