Benchmarks and comparisons with decision models

Independent benchmark of Jev vs Sonnet 5, GPT-5 nano and local LLMs on 770 Reddit AITA verdicts: Brier score, latency and cost.

Study of whether Jev's reading of financial news predicts stock returns: strong same-day signal, no tradeable alpha.

Benchmark of Jev versus Claude Haiku 4.5 as input safety classifiers on jailbreak, evasion and benign pseudo-harmful prompts.

Experiment using Jev through langchain-typesafe as an agent-eval judge, compared with GPT-5.6 and Claude Sonnet 4.6 against human labels.

Two Jev benchmarks with a playground: chess (out of lane, about 950 Elo with code-supplied facts) and NPC addressee detection (F1 0.96).

Reproducible benchmark of Jev vs Claude Sonnet 5 on classifying agent tool calls as readonly, destructive, privileged or exfiltration.

Large rubric-classification benchmark of Jev vs Claude Haiku 4.5, Sonnet 5, OpenJev and many open models, with full price tracking.

Probability-aware evaluation of decision models: calibration, coverage at a fixed error budget and latency, Jev vs GLiNER2.5.

Benchmark of zero-shot Jev versus a PyTorch MLP and k-NN on embeddings for Jigsaw toxic comment classification.

Independent evaluation of Jev on MetaTool, When2Call and BFCL V4 tool selection, abstention and tool-use decisions.

1,000-decision benchmark of Jev 1.13 across 25 deterministic reasoning families, showing where it excels and where it fails.

Reproducible benchmark comparing Jev with hosted and open decision models on rule application, probabilities, latency and cost.

Reproducible 2,976-request measurement of whether batching questions in one Jev call bills the state once and keeps answers stable.

100-question benchmark placing Jev's reasoning at 77% against small Qwen3.5 models under different thinking budgets.

Benchmark asking Jev to pick the winner of 10,984 real Upworthy headline A/B tests: 64.5% overall, 74.7% when decisive.

100-question-per-cell check of Jev on Korean versus English reading, paraphrase and medical exams, with recorded responses.

Awesome-style map of Jev resources plus independent, reproducible benchmarks of primitives, confidence gating and fan-out latency.

Local benchmark comparing Jev with Laya-MLX on classifying GitHub issues for beginner-friendliness, CV fit and difficulty.

Speed and accuracy benchmark of hosted Jev 1.13 versus local Laya 421M on 1,470 synthetic items across eight decision tasks.

Pre-registered benchmark of Jev as a medical hallucination check on MedHallu, versus fast LLMs, on accuracy, latency and cost.

Jev-style medical QA on Qwen3.5-4B that picks the answer option in one forward pass instead of generating text.

Reproducible benchmark of Jev on Brazil's ENEM 2025 exam, measuring accuracy and calibration with an abstain option against open LLMs.

Independent test of whether Jev's probabilities stay calibrated on 900 unseen rule-generated tickets plus three public benchmarks.

Measures whether SQL ORDER BY over Jev probabilities gives a defensible ranking: inversions, ordinality, calibration, wording and ties.

Benchmarks Jev against local open decision models on 480 general Persian questions, literary questions and a classical poetry pilot.

Reproducible Turkish RAG benchmark comparing Jev batch-Noul reranking with Cohere Rerank on quality, latency and cost.

Reproducible Chinese RAG reranking benchmark for Jev on T2Reranking with hard negatives, staged and cost-capped, logs published.

Benchmark for agent regression after config edits: a Jev Choice decides which approved answers changed meaning versus wording.

Reproducible evaluation of Jev 1.13 as a reward model, LLM judge and process verifier across eight benchmark tracks.

Jev as an LLM router on RouterArena and LLMRouterBench: the router beat the best single model, but a no-Jev ablation matched it.