nielsr/coref-roberta-large

CorefRoBERTa is a transformers model pretrained on a large corpus of English data in a self-supervised fashion . It was pretrained with two objectives: Masked Language Modeling (MLM) and Mention Reference Prediction (MRP) objectives . This is different from traditional neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like the future tokens . It allows the model to learn a bidirectional representation of the entire sentence . If you have a dataset of labeled sentences for instance, you can train a standard classifier using the features produced by the Coref RoBERTa model as inputs as inputs .,