claude-referee
Unofficial Claude Code plugin that checks 'done' claims and small judgment calls with Jev, logs receipts, and can warn on unverified stops.
Overview
claude-referee checks the fluent claims coding agents make, such as "All tests pass. Done.", along with small judgment calls like which option fits the project's rules. Claude still makes the big decisions; checkable questions go to Jev, which answers with a probability, and every check is logged locally as a receipt.
Commands done, decide, judge and claims ask Jev and print a one-line answer, and sessions start with a short usage note for Claude. A stop gate is off by default: shadow mode only records what it would have done, soft mode adds a warning, and an active blocking mode is planned but not yet recommended; it fails open if Jev is slow or down. The README reports measurements: reordering options moved Jev's probability by up to 0.52 while repeating the same question moved it by at most 0.01, so decide asks every choice question in several orders. It also notes the real cost of a check is Claude's time, not Jev's. Docs include a Turkish manifesto.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this 'done' claim actually verified? | yes · no |
| Choice | Which of these options fits our rules? | — |
How it works
- User or Claude runs done, decide, judge or claims
- Jev answers (decide asks in several option orders)
- Receipt is logged locally
- Optional stop gate records, warns or (planned) blocks
Strengths
- Mitigates option-order sensitivity
- Local receipts for every check
- Gradual rollout modes
Caveats
- Active blocking mode not yet recommended
- Unofficial
Getting started
Install the plugin from the repository in Claude Code and set your TypeSafe key.At a glance
- Type
- Plugin
- Status
- beta
- Runtime
- Claude Code plugin
- Has code
- yes
Use cases and integrations
How curated lists describe it
Evidence over eloquence. Unofficial Claude Code plugin that checks "done"…
wh000wh000/awesome-jev-liveEvidence over eloquence. Unofficial Claude Code plugin that checks "done" claims and small judgement calls with TypeSafe Jev, and keeps receipts.
hellogumbo/awesome-jev
Frequently asked questions
Will it block Claude from stopping?
Not by default; shadow and soft modes only record or warn.
Why ask choices in several orders?
Measurements showed option order moved Jev's answer far more than repetition.
What does a check cost?
The README says the main cost is Claude's turn time, not Jev.
Related projects
Agent skills for placing, building, evaluating and improving decision-model systems, with Jev as the default hosted example.
Agent skillbetaClaude Code plugin and CLI for the Jev funnel: judge items in bulk, settle confident ones in code, send only uncertain ones onward.
PluginbetaPythonEval-first MCP server for Jev: prototype questions, map them over many items, and measure accuracy and thresholds on labelled data.
MCP serverbetaTypeScriptAgent skill that tells you when you need an eval and how to build it, with an opt-in lane routing closed-label residue to Jev.
BenchmarkbetaPythonPre-registered audits of AI judges that measure how many decisions a judge can take alone at a given error rate, with calibration.
BenchmarkbetaPythonClaude Code and Codex skills powered by Jev, published with a calibration curve measured on 4,995 real agent decisions.
Project