aliosm/ai-soco-cpp-roberta-tiny

From scratch pre-trained RoBERTa model with 1 layers and 12 attention heads using AI-SOCO dataset . Can be used to do code classification, authorship identification and other downstream tasks on C++ programming language . Model can be used directly after tokenizing the text using the provided tokenizer with the model files . The model is limited to C++ language only . It is only available for C++ code classification and authorship ID on the C++ model . It was trained on Google Colab platform with 8 TPU cores for 200 epochs, 32*8 batch size, 512 max sequence length and MLM objective . Each continues 4 spaces were converted to a single tab character (,