LlamaIndex Jev
Unofficial LlamaIndex reranker and router where Jev scores retrieved passages and selects query engines, with BEIR results.
Overview
LlamaIndex Jev provides two drop-in LlamaIndex components powered by Jev. JevRerank is a node postprocessor that scores each retrieved passage on a 0 to 3 relevance rubric (off-topic to fully answers) rather than cosine similarity and keeps the top n. JevSingleSelector and JevMultiSelector choose which query engine or tool should handle a query in a RouterQueryEngine. Reranking fails open, keeping retrieval order on errors, while selection fails closed, raising or using a declared default.
It works with the TypeSafe API, OpenRouter, Vercel AI Gateway or Cloudflare by setting a provider. On the BEIR nfcorpus test with 323 queries, adding JevRerank after MiniLM dense retrieval raised nDCG@5 from 0.340 to 0.396 (BM25 was 0.298), and a listing puts the cost at about $0.0003 per query. It is published as two PyPI packages with docs and runnable examples that use mock embeddings so no OpenAI key is needed. It is an independent community project.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Score | How relevant is this passage to the query (0-3, off-topic to fully answers)? | 0 · 1 · 2 · 3 |
| Choice | Which query engine or tool should handle this query? | — |
How it works
- Retriever returns candidate passages
- JevRerank scores each and keeps the top n
- Or a Jev selector chooses the query engine
- Query engine answers with the selected context
Strengths
- Measured nDCG gain on a standard benchmark
- Several provider options
- Sensible failure modes: rerank fails open, select fails closed
Caveats
- Unofficial
- One benchmark dataset reported
- Each query adds an API call
Getting started
pip install llama-index-postprocessor-jev llama-index-selectors-jev, export TYPESAFE_API_KEY, then add JevRerank(top_n=5, mode="score") as a node postprocessor.At a glance
- Type
- Library
- Status
- beta
- Language
- Python
- Runtime
- LlamaIndex packages
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Retrieval / RAG: unofficial LlamaIndex adapter where Jev `Score`s each retrieved passage and `Choice`/`Noul` selects the query engine, with nfcorpus nDCG@5 0.340→0.396 at about $0.0003/query.
yibie/awesome-jevUnofficial LlamaIndex reranker (`JevRerank`) and router (`JevSingleSelector` / `JevMultiSelector`) on the official Python SDK
AnotiaWang/awesome-decision-modelsProvides Jev rerankers and routers for LlamaIndex to score retrieved passages or select query tools.
logicrw/awesome-jev-projectsLlamaIndex reranker and selector using Jev Score and Choice answers, with configurable confidence handling.
cobanov/awesome-jevLlamaIndex reranker + router powered by TypeSafe Jev — typed scores/choices, cheaper than LLM-as-judge.
hellogumbo/awesome-jev
Frequently asked questions
How much does reranking help?
On BEIR nfcorpus, nDCG@5 rose from 0.340 to 0.396 over MiniLM alone.
What happens if Jev fails?
Reranking keeps the original order; selection raises or uses a declared default.
Which providers are supported?
TypeSafe, OpenRouter, Vercel AI Gateway and Cloudflare.
Related projects
Drop-in rerank API that speaks Cohere, Jina and Voyage request shapes and scores each document with a Jev relevance question.
ServicebetaJavaScriptRails gem for intent search: Jev answers declared questions when records are saved, and searches become weighted SQL over those answers.
ProjectbetaRubyPiJev: Pi-based coding agent where Jev shortlists skills, ranks code excerpts and triages tool failures while your model writes code.
ProjectprototypeTypeScriptMilvus's model library with a Jev reranker that sends candidate documents as yes/no relevance questions and returns them sorted.
LibraryproductionPythonComposio agent SDK with an optional TypeSafe provider that uses Jev to shortlist and pick tools and closed-set arguments.
ProjectproductionTypeScriptPython multi-agent runtime with native Jev support: agents ask typed questions via a decision client separate from the text model.
LibraryproductionPython