Evaluation & Benchmarking

Single-script CLI and library for NLI classification and multiple-choice reranking with the AlexWortega/openjev Qwen3.5-4B cross-encoder.

Localhost teaching aid that uses Jev to estimate where instructional text asks readers to hold too many ideas at once.

Novelty demo that turns Jev answers to 22 Boolean gate questions into lookup tables for a simulated RISC-V CPU running compiled C.

Local PR reviewer: deterministic diff handling plus Jev or Laya judgments of severity, category and true-positive per hunk.

Portuguese workshop: a LangGraph sales agent where each decision step can be made by Jev or a frontier LLM and measured side by side.

Benchmark testing whether Jev or open Kev can replace an LLM judge in agent guardrail callbacks on 7,925 public examples.

Study of Jev's choices in multi-armed bandits, testing whether Bayesian instructions and supplied statistics improve its exploration.

Experiments testing whether Jev can spot positive-expected-value dice bets from payout tables, and whether recent wins or losses sway it.

GoodWatch movie and TV recommendation monorepo, listed for an experiment comparing Jev question designs for attribute scoring.

Zero-shot run of Jev 1.13 on all seven LexGLUE legal tasks (23,607 examples) versus GPT-5.6 Luna, open models and fine-tuned BERT.

Pre-specified study running Jev and GPT-4.1 as 300 synthetic survey respondents; question format mattered more than model.

LlamaIndex notebook comparing Jev with open decoders, encoders and specialised tools on document-pipeline Choice tasks.

Pre-registered paired audit of Jev on Spanish across 3,200 labelled items, plus a CLI to rerun it on your own data.

FGV Direito SP experiment comparing Jev with Gemini 3.8 Flash and GPT-5.6 Luna for annotating 12 variables in 120 TJSP court rulings.

Pre-registered independent evaluation of Jev on intent classification and confidence-gated cascades, with a transparent erratum.

Independent field guide to Jev 1.13.0 behaviour across 11,621 requests: framing sensitivity, Snake and 3D city driving experiments.

Reliability-curve study of Jev on 240 hand-labelled agent tool calls, publishing labels, raw responses and scripts.

Pre-registered study testing Jev's calibration claims on clinical diagnosis cases against Claude Opus 5 and Opus 5.5.

Jev applied to SemEval-2026 DimABSA dimensional sentiment tasks using only typed primitives, no generation or fine-tuning.

Benchmark harness comparing Jev with OpenRouter models and open Jev-shaped models on accuracy, calibration, latency and cost.

Write-up testing whether Jev beats small fast LLM tiers on public and private classification, as a classifier or a filter.

Third-party check of Jev versus GPT-4o-mini and Claude Sonnet 4.5 routing 60 multilingual photo-shoot booking inquiries.

Agent harness with 100 mocked tools measuring steps to finish tasks when the LLM picks tools versus when Jev picks them.

Experiments testing Jev ideas against baselines, starting with Jev as a top-10 tool router for MCP agents on three benchmarks.

Audit of 300 web pages for AI citability with nine Jev decisions each, checked against an LLM judge for agreement, cost and latency.

Nine-board gomoku workbench comparing how two Jev players respond to five different input representations of the board.

Same support agent and 25 rules guarded twice, by an LLM judge and by Jev, compared on accuracy, latency, cost and coverage.

Evaluation of Jev versus DeepSeek and Qwen on healthcare decision tasks: triage, diagnosis, medication, nursing and trial screening.

Reproducible measurements of Jev's real cost, latency, response shapes and accuracy, plus 13 decision models on the same 633 items.

Independent 52-page Chinese report on Jev that checks vendor claims and reproduces 50 Chinese tests, with traceable data.