iarfmoose/bert-base-cased-qa-evaluator

The QA evaluator is a pretrained BERT-base-cased with a sequence classification head . The model is trained to evaluate if a question and answer are semantically related, but cannot determine whether an answer is actually true/correct or not . The training data was made up of question-answer pairs from the following datasets: SQuAD-RACE, CoQA, MSMARCO and MSQA. The model was then trained to predict whether the input sequence represented one of the original QA pairs or a corrupted input . The data was used only 50% of the time a corruption operation was performed (either swapping the answer for an unrelated,