nielsr/tapas-base

TAPAS is a BERT-like transformers model pretrained on a large corpus of English data from Wikipedia in a self-supervised fashion . The model was pre-trained on MLM and an additional step which the authors call intermediate pre-training . It uses relative position embeddings by default (i.e. resetting the position index at every cell of the table) The training examples are created based on synthetic as well as counterfactual statements . It can then be used to extract features useful for downstream tasks such as answering questions about a table, or determining whether a sentence is entailed or refuted by the contents of a table . See the model hub to look for,