Evaluation & Benchmarking

Independent benchmark of Jev versus GPT-4.1-mini on 200 South African customer enquiries, showing how question wording changed results.

Benchmark of Jev versus rules, local PII models and LLMs at finding personal information: accuracy, leaks, cost and latency.

Open CC-BY-4.0 data behind jevals.com: leaderboards and per-decision logs grading Jev and LLMs on accuracy, calibration, cost and speed.

Small reproducible benchmark of Jev against LLMs, fine-tuned BERT, Laya and zero-shot NLI on SST-2, AG News and Banking77.

Open framework and preprint benchmarking typed decision models such as Jev, pplx-decider, Liquid D1 and Clef in six clusters.

Metamorphic benchmark testing whether decision-model probabilities stay coherent under 50 laws, with no gold labels.

Benchmark of nine local Jev-style decision engines on Apple Silicon across English and Korean sets, with paired significance tests.

10,000-sample multimodal benchmark for decision models that return full probability distributions over declared candidates.

Tutorial to fine-tune Qwen3.5-4B into a Jev-like classifier with Unsloth on RunPod, serve via vLLM and benchmark against Jev.

Experimental harness for typed JSON decisions from DiffusionGemma by picking the likeliest allowed tokens from final logits.

Public benchmark of PadFlow's land-development SaaS decisions: schemas, anonymised seed rows and a runner for calibrated decision models.

Benchmark comparing Privatemode (GLM-5.3-Flash), TypeSafe Jev and Laya on speed, accuracy, calibration and cost.

Auditable reranking benchmark: Jev vs Qwen3-Reranker-0.6B on BEIR SciFact and NFCorpus for nDCG, cost per query and tail latency.

Paper and benchmark testing Jev as a zero-shot detector of AI alignment failures across 44 benchmarks and ten failure types.

Benchmark of Jev versus OpenAI structured outputs extracting smoking history from 1,000 synthetic notes.

Reproducible benchmark of SubQ Snap, TypeSafe Jev and three open decision models on software-engineering decisions.

Benchmark of Jev on 159 community-reconstructed questions from Indonesia's SNBT 2025 university entrance exam, all responses published.

Independent head-to-head benchmark of System One decision models on one sealed manifest, from Jev and Laya to 49 open-weight entries.

Benchmark where Jev steers an agent through 100 key-and-door grid puzzles, one Choice per move, scored against an exact BFS solver.

Benchmark of LLM-native structured output (Qwen on Cerebras) versus Jev on seven application tasks: errors, latency, tokens and cost.

Browser-agent benchmark comparing WebMCP with screenshots, DOM and code execution, including a Jev + Mercury 2.5 configuration.

Independent benchmark asking whether Jev can replace an LLM reranker, on natural-language search over 6,245 YC companies, with a live demo.

Benchmark and web UI comparing dozens of zero-shot extraction and classification systems, including Jev-like engines.

First Chinese calibration benchmark for Jev-class models: accuracy, ECE, option order and script robustness.

Hugging Face Space indexing open reproductions of Jev and comparing them on a versioned decision benchmark with calibration metrics.

Experiment testing how well Jev predicts financial markets; the author reports it does not do well, like most LLMs.

Side-by-side comparison of Jev and DeepSeek-flash on speed, tokens, cost and accuracy for extraction, classification and reranking.

Personal experiment repository for trying the jev-latest decision model.

Microsoft's hosted decision model announcement, claiming top accuracy on 36 benchmarks and low latency versus GPT-6 Sol.

Retrospective test of Jev 1.13 classifying 8,054 Kepler Objects of Interest against hidden NASA Exoplanet Archive dispositions.