Evaluation & Benchmarking

Minimal web demo: paste an agent step trace and Jev judges whether to continue, retry, escalate or stop.

Small demo collection testing Jev's spatial reasoning with maze single-step choices versus multi-step lookahead.

Japanese demo judging the same utterance with Jev and an LLM to compare emotion-shift outputs and speed side by side.

Upload a paper PDF and get a fast Jev-based judgment of understandability and completeness, with low-confidence answers flagged.

Local single-page playground for Jev with templates from 37 benchmark datasets, showing gold answers next to typed results.

Self-hosted playground running SystemOne requests through an emulator, Jev and an LLM judge to compare per-question fidelity.

Minimal web playground to write freeform state, build custom Choice, Score and Noul questions and see live Jev probability distributions.

Browser-only Jev testbed via Vercel AI Gateway for building boolean, choice and score questions and viewing probabilities and cost.

Chinese 'emotion lie detector' demo comparing Jev's Choice, Score and Noul judgments on a message with a general LLM.

Local Python toolkit and web dashboard for trying Jev via OpenRouter or TypeSafe, with review analysis, topic routing and request history.

Video demo code comparing Jev with Laya, Strands Decider and Claude on grading, fetching and prompt-injection resistance.

Experiment testing whether Jev knows music theory via SATB part-writing and quizzes vs Claude, plus a Jev-driven Rhodes piano demo.

MoonBit and TypeScript Jev playground: gomoku, shell-command risk gate, ESLint plugin, a tiny Jev-condition language and measured reports.

Experiment screening ADHD review abstracts as include or exclude with Jev, scored against Cohen et al. 2006 abstract triage labels.

Measurement-first lab deciding which classifier technique, Jev, rules or other models, earns a place in a coding-agent fleet.

50 small interactive demos comparing Jev's Choice, Score and Noul with OpenAI gpt-4o-mini side by side, runnable offline from fixtures.

Video demo classifying 50 mock YouTube comments with Jev (intent, value, video idea) and comparing speed and cost with GPT-4o-mini.

Hypothesis-driven experiments characterising the Jev API, intended to inform a future football prediction product.

Comparative judgment of ASAP essays by Jev and two LLM judges, turning pairwise decisions into scores compared with human grades.

Learning experiment comparing Jev with an OpenAI model on GitHub issue ownership and on AG News, TREC and SNIPS classification.

Independent measurements of Jev versus open Laya on typed decisions, plus a local-first cascade that matches Jev at 1.8x speed.

Reproducible test of SubQ Snap and Jev on 3,840 questions about a fictional store's event log from 2K to 360K tokens of context.

MIT-licensed benchmark and leaderboard for Jev-class decision models: accuracy, calibration, latency and cost on sealed tiers.

Open benchmark for System One decision models on Choice, Score and Noul, reporting accuracy, calibration and robustness separately.

Convex's eval repo with a 108-question multiple-choice benchmark comparing Jev's Choice answers with 14 LLMs on accuracy, latency and cost.

Reproduction kit for the Decision Index leaderboard: runs 37 public benchmarks against any /v1/systemone engine.

System One Mosaic Benchmark comparing Jev and open decision models across 137 English and 27 Japanese Choice, Noul and Score tasks.

Agent skill that tells you when you need an eval and how to build it, with an opt-in lane routing closed-label residue to Jev.

Python tool that measures where to route decisions between Jev and a larger model on your own data, then routes by confidence.

Standard-library Python workbench for repeatable classifier latency and accuracy experiments across Jev, Laya and classifier.dev models.