cross-encoder/ms-marco-TinyBERT-L-2

The model uses BERT-Tiny, a tiny BERT model with only 2 layers, 2 attention heads and 128 dimension size . It was trained on MS Marco Passage Ranking task . The model can be used for Information Retrieval: Given a query, encode the query will all possible passages (e.g. retrieved with ElasticSearch) Then sort the passages in a decreasing order . The training code is available here: SBERT.net Training MS Marco. For more details, see the training code and performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset . Pre-trained models can also be used like this: Pre- trained models can be,