Cribrix
Precision-first RAG service where Jev routes queries, triages each retrieved chunk and verifies every claim before an answer is released.
Overview
Cribrix is a self-hosted retrieval-augmented generation orchestrator built so that the pipeline is allowed to say no. It wraps a conventional LLM (System 2) in a loop gated on both sides by Jev (System 1). Jev first routes the query as search or chitchat; then, for every retrieved chunk, one Jev call returns a relevance Score plus Noul checks for whether the chunk answers the question and whether it contains a prompt injection. If too little context survives, the LLM is never called.
After generation, one more Jev call asks a Noul per claim in the draft, and code also checks that numbers in the answer appear in the sources. Drafts with unsupported claims are withheld, and each refusal has a distinct status such as NO_DOCUMENTS, INSUFFICIENT_CONTEXT, DECLINED or UNGROUNDED. The README reports, on a replayed 62-question golden set, 0 of 22 unanswerable questions answered versus 4 of 22 for naive top-5 RAG, with CI gating merges on these recorded results.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Is this query a search or chitchat? | SEARCH · CHITCHAT |
| Score | How relevant is this chunk to the query? | — |
| Yes / no | Does this chunk answer the question / contain a prompt injection? | yes · no |
| Yes / no | Is this claim in the draft supported by the sources? | yes · no |
How it works
- Jev routes the query; chitchat is answered without retrieval
- pgvector retrieves top 20 chunks; one Jev call per chunk scores relevance and checks evidence and injection
- If not enough context survives, return INSUFFICIENT_CONTEXT without calling the LLM
- LLM drafts an answer; Jev checks each claim and code checks numbers; ungrounded drafts are withheld
Strengths
- README reports zero unanswerable questions answered on its golden set
- Avoids LLM calls when context is insufficient
- Distinct refusal statuses for observability
- Reproducible offline evaluation from recorded calls
Caveats
- Golden set is small (62 questions) and author-built
- Requires TypeSafe and an LLM provider
- Self-hosted Postgres with pgvector needed
Getting started
See the README; make eval replays the recorded Jev and LLM calls offline.At a glance
- Type
- Service
- Status
- beta
- Language
- Python
- Runtime
- self-hosted service (Postgres + pgvector)
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Retrieval / RAG: filters retrieved chunks with a Jev `Score` plus `Noul` checks for answer evidence and prompt injection, then withholds any draft whose claims fail a batched per-claim `Noul` or cite numbers absent from the sources; on its replayed 62-question golden set it answered 0 of 22 unanswerable questions, against 4 of 22 for naive top-5 RAG.
yibie/awesome-jevSelf-hosted RAG service that uses Jev to decide which retrieved context reaches the generator and whether the generated answer is released.
logicrw/awesome-jev-projectsSelf-hosted RAG service where Jev picks search or chitchat with a Choice, answers a relevance Score plus Noul checks for answer evidence and prompt injection on each retrieved chunk, and answers one Noul per claim in the generated draft; code applies the thresholds, refuses without calling the LLM when no chunk survives, and withholds drafts with unsupported claims or numbers absent from the sources.
cobanov/awesome-jev
Frequently asked questions
What does Jev decide in Cribrix?
Query routing, per-chunk relevance, evidence and injection checks, and per-claim support for the generated draft.
How is it different from naive RAG?
It refuses when context is missing and withholds drafts with unsupported claims instead of always answering.
Can I reproduce the results without keys?
Yes, make eval replays committed recordings of live calls offline.
Related projects
jevctl: Node CLI for typed Jev judgments (verify claims, screen agent hijacks, classify) with exit codes for CI gating.
CLIbetaTypeScriptTyped-decision accuracy layer for RAG: six checks for intent, relevance, clarification, sufficiency, grounding and caveats.
ServicebetaPythonEvidence gate for RAG: a System One model judges whether retrieved excerpts support an answer before the LLM is allowed to speak.
MCP serverbetaPythonMCP server giving coding agents typed judgment tools (verify, screen, rerank, gate, score) backed by Jev, with auto/review/escalate policy.
MCP serverbetaPythonSmall stdio MCP server with ask, rank and verify tools backed by Jev, a local event log and installers for several agent harnesses.
MCP serverbetaTypeScriptPermission-aware document library using Jev or Cloudflare Clef for hierarchical search, passage scoring and automatic filing.
LibrarybetaTypeScript