dsh-jev-verify
DeepSeek Harness plugin with jev_decision tools, an opt-in risk and loop auto-guard, and a live self-test benchmark against the Jev API.
Overview
dsh-jev-verify makes Jev a first-class plugin for DeepSeek Harness and is built to be verified rather than trusted. jev_decision asks Choice, Score and Noul questions against a state in one call and returns typed answers, confidence, probabilities, token usage, measured latency and estimated cost. jev_verify runs a built-in labelled benchmark of 27 questions, including guard verdicts, against the live API and reports measured accuracy, latency percentiles, calibration and cost. jev_guard_status audits the guard.
The opt-in auto-guard first applies a free deterministic blacklist to shell-like tool calls (rm -rf /, disk formatting, dropping databases, credential exfiltration); risky-looking commands that pass are judged by a Jev risk noul and denied above a threshold, failing open with a warning on API errors. A loop guard detects semantic stalls in repeated calls and injects advice. There is no mock mode or silent fallback: without a key, tools fail with setup instructions.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this shell command risky enough to deny? | yes · no |
| Yes / no | Are these repeated tool calls a semantic stall?(inferred) | yes · no |
How it works
- Agent calls jev_decision with typed questions
- Optional auto-guard: blacklist, then Jev risk noul for shell calls
- Loop guard advises on repeated stalls
- jev_verify measures live accuracy, latency and calibration
Strengths
- Built-in live self-test
- Every call reports latency, tokens and cost
- No silent fallbacks
Caveats
- Guard fails open on API errors
- Small 27-question benchmark
Getting started
Install as a DSH plugin, set TYPESAFE_API_KEY, and run jev_verify to measure the live API.At a glance
- Type
- Benchmark
- Status
- beta
- Language
- TypeScript
- Runtime
- DeepSeek Harness plugin
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Jev (TypeSafe System One) decision tools + live verification benchmark for DeepSeek Harness: jev\_decision (choice/score/noul) and jev\_verify, honest by design.
logicrw/awesome-jev-projectsJev (TypeSafe System One) decision tools + live verification benchmark for DeepSeek Harness: jev_decision (choice/score/noul) and jev_verify, honest by design.
hellogumbo/awesome-jev
Frequently asked questions
What is jev_verify?
A built-in benchmark that measures accuracy, latency, calibration and cost against the live API.
Is the guard on by default?
No, auto-guard is opt-in.
Is there a mock mode?
No, it deliberately has no mock or silent fallback.
Related projects
Plugin suite adding Jev judgments to DeepSeek Harness: tool pruning, dead-loop blocking, a safety guard and skill routing.
PluginbetaTypeScriptSupervisor for unattended coding agents: one batched Jev Choice over the trajectory decides continue, pause or escalate.
ProjectbetaPythonStop hook for coding agents where Jev checks each attempted stop against plain-language rules and sends the agent back if one is violated.
HookbetaPythonHackathon arena where 200 people attack a support agent and a Jev allow/block/escalate gate on every tool call is hardened each round.
ProjectprototypePythonPreliminary SHADE-Arena study of Jev as a monitor and per-action gate for covert agent sabotage, compared with Gemini 2.5.
ProjectresearchPythonDeepSeek Harness plugin where Jev classifies risk, irreversibility, task fit and injection for every tool call before it runs.
PluginbetaTypeScript