jev-rag-guard
Typed-decision accuracy layer for RAG: six checks for intent, relevance, clarification, sufficiency, grounding and caveats.
Overview
rag-guard (repository jev-rag-guard) wraps the seam between retrieval and generation in a RAG pipeline with six small typed checks and a full decision trace. It does not retrieve or generate itself. Intent checks before retrieval whether a query should be escalated or refused; relevance drops passages that do not help; clarify stops to ask when the answer depends on missing information; sufficiency skips generation and returns an honest 'I don't know' when context is not enough; grounding flags claims unsupported by the context; and caveat flags answers that depend on a condition that must be surfaced.
Each check is a single bounded yes/no or closed-set judgment with a probability that can be thresholded and logged. The decision model is pluggable: the core has zero dependencies and includes a lexical heuristic backend that needs no key, with optional OpenAI, Anthropic and TypeSafe Jev backends. Code owns thresholds and routing. A SKILL.md is included, and CI runs tests.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Does this query need to be escalated to a human or refused?(inferred) | proceed · escalate · refuse |
| Yes / no | Does this passage help answer the query? | yes · no |
| Yes / no | Is the kept context enough to answer confidently and completely? | yes · no |
| Yes / no | Is each claim in the answer supported by the context? | yes · no |
How it works
- Check query intent before retrieval
- Filter retrieved passages by relevance
- Ask for clarification or check sufficiency before generating
- Check grounding and caveats after generation
- Return the answer with a full trace
Strengths
- Turns hallucination-prone steps into explicit checks
- Pluggable backends including a keyless heuristic
- Zero-dependency core
- Traceable decisions
Caveats
- Heuristic backend is far weaker than a model
- Thresholds need tuning
- Adds several calls per query with a model backend
Getting started
pip install -e . (add .[typesafe] for Jev), then follow the Quickstart.At a glance
- Type
- Service
- Status
- beta
- Language
- Python
- Runtime
- Library
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
RAG accuracy layer where Jev answers Noul and Choice questions to filter retrieved passages, check whether the retrieved context is sufficient to answer, and verify that generated claims are grounded in the sources; code owns the thresholds and routing, and a lexical heuristic backend runs with no API key.
cobanov/awesome-jev
Frequently asked questions
Does it need Jev?
No. Jev is one pluggable backend; a lexical heuristic works without any key.
What happens when context is insufficient?
Generation is skipped and an honest 'I don't know' is returned.
Does it hide unsupported answers?
No. Grounding flags them rather than hiding them.
Related projects
Precision-first RAG service where Jev routes queries, triages each retrieved chunk and verifies every claim before an answer is released.
ServicebetaPythonEvidence gate for RAG: a System One model judges whether retrieved excerpts support an answer before the LLM is allowed to speak.
MCP serverbetaPythonMCP server giving coding agents typed judgment tools (verify, screen, rerank, gate, score) backed by Jev, with auto/review/escalate policy.
MCP serverbetaPythonSmall stdio MCP server with ask, rank and verify tools backed by Jev, a local event log and installers for several agent harnesses.
MCP serverbetaTypeScriptPermission-aware document library using Jev or Cloudflare Clef for hierarchical search, passage scoring and automatic filing.
LibrarybetaTypeScriptPython library that replaces hard-coded RAG thresholds with calibrated decisions for stopping, chunking, selection, abstention and caching.
ServicebetaPython