jev-fidelity
Editorial QA tool where Jev judges per fact whether an edit preserved the original, with a confidence gate and human review fallback.
Overview
jev-fidelity is one of four apps in jev-suite, a Java collection of decision-quality tools that share a kernel. It answers whether an edit preserved the information in the original text: for each fact unit, Jev labels the edit as preserved, equivalent, drift or lost. A 0.70 confidence gate in code decides what passes, and anything uncertain degrades to human review rather than an automatic pass.
The shared jev-kit provides a typed client with retry handling for rate limits and server errors, a three-way gate (confident pass, confident fail, otherwise review), and a sanitiser that defends against prompt injection in external text. Suite rules: Jev judges and code decides, unreachable models escalate rather than pass, calibration runs refuse mock data, and missing evidence fails closed. The README reports a 55-sample, 110-fact calibration on real Wikipedia revision diffs with 91 of 92 gated judgements correct, 17 designed abstentions and 0 of 20 injection attempts flipping a result.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Did the edit preserve this fact from the original? | preserved · equivalent · drift · lost |
How it works
- Split original and edited text into fact units
- Sanitise external text and ask Jev a Choice per fact
- Apply the 0.70 confidence gate in Java
- Pass, fail, or route to human review
Strengths
- Degrades toward caution rather than auto-passing
- Injection-resistance tested
- Calibration on real revision diffs
Caveats
- Calibration set is small (55 samples)
- Version 0.1.0
At a glance
- Type
- Project
- Status
- beta
- Language
- Java
- Runtime
- Library and CLI
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Editorial QA: asks Jev per fact unit whether an edit preserved the original (preserved / equivalent / drift / lost) behind a 0.70 confidence gate in code, degrading to human review rather than pass; 55-sample calibration on real Wikipedia revision diffs reports 91/92 gated judgments correct and 0/20 injection flips.
yibie/awesome-jev
Frequently asked questions
What does Jev decide?
Whether each fact in the original was preserved, rephrased equivalently, drifted or lost in the edit.
What happens when Jev is unsure?
The case goes to human review; it is never auto-passed.
How accurate was it?
The README reports 91 of 92 gated judgements correct on a 55-sample calibration.
Related projects
Prose linter for AI writing tells: regex handles 18, Jev judges 15 as Nouls in one call, shipped as a Claude Code Stop hook.
HookbetaPythonRewrites AI drafts in your own voice, with Jev as the judge in a bounded loop that checks style and that meaning stays intact.
ProjectprototypePythonPython library that replaces hard-coded RAG thresholds with calibrated decisions for stopping, chunking, selection, abstention and caching.
ServicebetaPythonResearch implementation of Jev-style typed decisions over clinical evidence: supported, contradicted or unresolved.
ProjectresearchPythonPython harness stress-testing Jev's confidence, its accuracy as a RAGTruth hallucination judge, and its use as a RAG reranker.
DemoresearchPythonPublished Jev client and claim check from the Fake/Real evidence-review site, where Jev filters sources and judges each claim.
ProjectproductionTypeScript