Evaluation & Benchmarking

Blog benchmark testing Jev against Mistral Small and Gemini Flash-Lite at validating local event listings, with cost and latency.

Independent leaderboard grading Jev and six LLMs on the same typed questions against human labels, with open per-decision data.

Static benchmark dashboard comparing Jev with two OpenAI Luna settings on Japan's 2026 university Common Test.

LangWatch comparison of Jev against seven open models under 1B on 15 decision tasks, with intervals, latency and contamination checks.

Working paper placing a text judge beside a live trading system as a read-only signal, comparing Jev and Claude as blind headline judges.

TypeSafe's published eval methodology and per-model results for System One decision workflows.

Blog review of a scheduler's pre-publish check where Jev answers four yes/no questions per social caption, compared with blind Fable labels.

Independent early-access trial of Jev that measures its calibrated judgments and reports cost per decision.

LangChain post comparing Jev with LLM judges for agent evaluation on accuracy, repeatability, latency and cost.

OpenRouter benchmark of Jev 1.13 vs Claude Opus 5 on Banking77: 81.0% vs 84.4% accuracy at a fraction of the latency and cost.

Independent experiments on Jev for search stopping, memory reranking and multi-hop relation selection, with stated limits.

Study comparing one direct Jev question per row with 12–14 Jev-scored dimensions and fitted weights on three classification tasks.

Blog post testing Jev in 16,000 calls against two OpenAI models, covering where it wins, where it breaks and how to set thresholds.

Independent benchmark of Jev versus four LLMs on Banking77 routing and prompt injection, plus a gated cascade.

Boring Tools Kit's technical SEO audit cost study, where Jev ranks striking-distance fixes and content gaps by calibrated probability.

Reddit post benchmarking Jev against generative LLMs, a BERT classifier and LAYA on the same task.

An X post comparing Jev with a private fine-tuned classifier at matched precision.

X post by a WebMCP benchmark author reporting Jev plus a small LLM solved 100% of tasks at about 112x lower model cost.

X post reporting a measured catalog-search run where Jev reranking alone did not beat vector retrieval.

Chinese X post measuring Jev in a RAG pipeline over TypeSafe's docs: correct top-ranked answers rose from 7 to 17 of 20.

Vals AI's independent benchmark of Jev: matched a frontier model on claim verification at a fraction of cost, but ranked last on LegalBench.