iarfmoose/roberta-small-bulgarian-pos

The model can be used to predict part-of-speech tags in Bulgarian text . Since the tokenizer uses byte-pair encoding, each word in the text may be split into more than one token . Using the last token was found to slightly outperform predictions based on the first token . The training data is unfiltered text from the internet and may contain all sorts of biases . The model achieves 98% on the test set. The training procedure was trained for 5 epochs over the training set . The loss was calculated based on label predictions for the last POS-tag for each word. The data was trained on the UPOS tags from (UD_Bulgarian-BTB).,