JevAdvBench
Paper introducing an adversarial benchmark for decision models: one appended opinion flips 12.1% of Jev 1.13 decisions.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
JevAdvBench is an arXiv paper presenting what it describes as the first adversarial benchmark for typed decision models. According to the listing, it contains 812 typed questions and 9,744 single-edit attacks against the state those questions are asked about.
The headline finding reported in the listing is that on Jev 1.13, appending a single unverified opinion to the state flips 12.1% of decisions. The authors' recommendation is that the state passed to a decision model should be treated as untrusted input, which matters for any application that feeds user-supplied or retrieved text into Jev. The paper itself was not available for review here, so attack categories and per-type results are not summarised beyond the listing.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Measures how often single-edit attacks on the state flip typed decisions | — |
How it works
- Build 812 typed questions
- Generate 9,744 single-edit attacks on the state
- Measure decision flips under attack
Strengths
- Quantifies robustness to adversarial state
- Actionable guidance to treat state as untrusted
Caveats
- Paper not reviewed in full
- Results specific to Jev 1.13
At a glance
- Type
- Paper
- Status
- research
Use cases and integrations
How curated lists describe it
First adversarial benchmark for decision models: 812 typed questions and 9,744 single-edit attacks. On Jev 1.13, one unverified opinion appended to the state flips 12.1% of decisions, so the state should be treated as untrusted input.
AnotiaWang/awesome-decision-models
Frequently asked questions
What did the paper find?
One unverified opinion appended to the state flipped 12.1% of Jev 1.13 decisions, per the listing.
How large is the benchmark?
812 typed questions and 9,744 single-edit attacks.
What should developers do?
Treat the state sent to a decision model as untrusted input.
Related projects
Benchmark of Jev versus Claude Haiku 4.5 as input safety classifiers on jailbreak, evasion and benign pseudo-harmful prompts.
BenchmarkresearchPythonPaper testing Jev, Laya, Decider and Nimble against classifiers and LLM judges on prompt injection and risk screening.
PaperresearchLarge catalog of Claude Code agents, commands, hooks and MCPs, listed with community Jev mods for guardrails and subagent models.
CLIproductionJavaScriptMCP proxy and hooks that screen agent tool calls before they run and their results before the agent reads them, using Jev judgments.
MCP serverbetaTypeScriptXano backend template where Jev judges each agent tool call as safe, needs review or dangerous, and uncertain calls wait for a human.
ProjectbetaXanoScriptEffect-based safety gate for coding-agent shell commands: fast rules first, then Jev decides allow/deny plus eleven risk checks.
ProjectbetaTypeScript