Jev reranking is not a free win
X post reporting a measured catalog-search run where Jev reranking alone did not beat vector retrieval.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
This entry is an X post describing a measured search-reranking run over 33,047 catalog entries, 164 real queries and 9,831 graded pairs. According to the list summary, Jev reranking alone did not beat vector retrieval in that setting.
It is a useful counterpoint to benchmarks where Jev reranking helps, showing that results depend on the domain, candidate generation and how the question is framed. No further details were captured.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this catalog entry relevant to the query?(inferred) | yes · no |
How it works
- Retrieve candidates from a catalog
- Rerank with Jev
- Compare against vector retrieval on graded pairs
Strengths
- Real queries and graded pairs at reasonable scale
Caveats
- Post only; methodology not captured here
- Single domain
At a glance
- Type
- Discussion
- Status
- discussion
Use cases and integrations
How curated lists describe it
Search reranking: a measured run over 33,047 catalog entries, 164 real queries, and 9,831 graded pairs reports that Jev reranking alone did not beat vector retrieval.
yibie/awesome-jev
Frequently asked questions
What was found?
Jev reranking alone did not beat vector retrieval on this catalog search task.
How big was the test?
33,047 catalog entries, 164 queries and 9,831 graded pairs.
Where was it posted?
On X.
Related projects
Benchmark of BM25, embeddings, rerankers and Jev for picking the right MCP tool among 525, plus an experimental Jev search engine.
MCP serverresearchTypeScriptFourteen-dataset reranking study of Jev vs Cohere Rerank 4, zerank-2 and chat models, with raw responses and bootstrap intervals.
BenchmarkresearchPythonGraded relevance study over 9,831 labelled pairs: Jev reranking alone does not beat embeddings, but fused with them it is best.
BenchmarkresearchPythonReproducible Turkish RAG benchmark comparing Jev batch-Noul reranking with Cohere Rerank on quality, latency and cost.
BenchmarkresearchPythonReproducible Chinese RAG reranking benchmark for Jev on T2Reranking with hard negatives, staged and cost-capped, logs published.
BenchmarkresearchPythonAuditable reranking benchmark: Jev vs Qwen3-Reranker-0.6B on BEIR SciFact and NFCorpus for nDCG, cost per query and tail latency.
BenchmarkresearchPython