youscan/ukr-roberta-base

Ukrainian Roberta was trained with code provided in HuggingFace tutorial tutorial . Currently released model follows roberta-base-cased model architecture (12-layer, 768-hidden, 12-heads, 125M parameters) The model was trained on 4xV100 (85 hours) with model built-in-built model architecture . It follows model's 12-layer model architecture with 768 hidden and 12-headed parameters . The model is based on the data from the Ukrainian Wikipedia and OSCAR . It is currently being used to train the model on the current version of Roberta. It is available in the original repository of this repository. It has been used in the Ukrainian version,