Skip to content

Benchmarks and comparisons with decision models

Head-to-head studies and leaderboards that measure accuracy, calibration, latency and cost of decision models against LLMs or each other.

354 public projects · page 3 of 12

  • LlamaIndex notebook comparing Jev with open decoders, encoders and specialised tools on document-pipeline Choice tasks.

    ProjectresearchPython
  • Pre-registered paired audit of Jev on Spanish across 3,200 labelled items, plus a CLI to rerun it on your own data.

    CLIresearchPython
  • FGV Direito SP experiment comparing Jev with Gemini 3.8 Flash and GPT-5.6 Luna for annotating 12 variables in 120 TJSP court rulings.

    ProjectresearchPython
  • Black-box reverse-engineering archive studying how the Jev decision model behaves, with evidence matrices and calibration analysis.

    ProjectresearchPython
  • Pre-registered independent evaluation of Jev on intent classification and confidence-gated cascades, with a transparent erratum.

    LibraryresearchPython
  • Independent field guide to Jev 1.13.0 behaviour across 11,621 requests: framing sensitivity, Snake and 3D city driving experiments.

    ProjectresearchPython, TypeScript
  • Reliability-curve study of Jev on 240 hand-labelled agent tool calls, publishing labels, raw responses and scripts.

    ProjectresearchPython
  • Independent API-only audit of what Jev's probabilities mean: abstain removal, Korean, coherence, option order, interference and caching.

    ProjectresearchPython
  • Research decoder that treats Jev as a language model, assembling replies from repeated choices over phrases, words and letters.

    ProjectresearchPython
  • Reproducible test of viral Jev HFT claims: direction not calibrated, never under 100 ms, and no edge after trading fees.

    ServiceresearchPython
  • Pre-registered study testing Jev's calibration claims on clinical diagnosis cases against Claude Opus 5 and Opus 5.5.

    ProjectresearchPython
  • Paper code where Jev screens agent tool output for relevance and prompt injection before the agent reads it.

    ProjectresearchPython
  • Research framework (DART) testing whether Jev's probabilities are a reliable signal to answer, escalate or abstain.

    ProjectresearchPython
  • Jev applied to SemEval-2026 DimABSA dimensional sentiment tasks using only typed primitives, no generation or fine-tuning.

    ProjectresearchPython
  • Reads Jev-style choice, yes/no and score probabilities from masked diffusion language models with shared Yes/No weights.

    ProjectresearchPython
  • Benchmark harness comparing Jev with OpenRouter models and open Jev-shaped models on accuracy, calibration, latency and cost.

    ProjectresearchPython
  • Write-up testing whether Jev beats small fast LLM tiers on public and private classification, as a classifier or a filter.

    ProjectresearchPython
  • Third-party check of Jev versus GPT-4o-mini and Claude Sonnet 4.5 routing 60 multilingual photo-shoot booking inquiries.

    ProjectresearchTypeScript
  • Agent harness with 100 mocked tools measuring steps to finish tasks when the LLM picks tools versus when Jev picks them.

    ProjectresearchTypeScript
  • Experiments testing Jev ideas against baselines, starting with Jev as a top-10 tool router for MCP agents on three benchmarks.

    ProjectresearchPython
  • Evidence ledger auditing Jev claims, recomputing published calibration and running an 800-item difficulty-gradient study.

    AppresearchPython
  • Audit of 300 web pages for AI citability with nine Jev decisions each, checked against an LLM judge for agreement, cost and latency.

    ProjectresearchPython
  • Nine-board gomoku workbench comparing how two Jev players respond to five different input representations of the board.

    ProjectresearchTypeScript
  • Same support agent and 25 rules guarded twice, by an LLM judge and by Jev, compared on accuracy, latency, cost and coverage.

    ProjectresearchTypeScript
  • Evaluation of Jev versus DeepSeek and Qwen on healthcare decision tasks: triage, diagnosis, medication, nursing and trial screening.

    ProjectresearchPython
  • Word-level language model whose output layer is Jev: next-word choices, Noul-verified drafted chunks and bits-per-token evaluation.

    ProjectresearchTypeScript
  • Reproducible measurements of Jev's real cost, latency, response shapes and accuracy, plus 13 decision models on the same 633 items.

    Serviceresearch
  • Runnable yardstick that tests Jev's three headline claims on a small local model: typed output, no hallucination, calibration.

    ProjectresearchPython
  • Simulation-only Payment Decision Lab testing whether Jev can beat rule-based routers at choosing payment service providers.

    ProjectresearchTypeScript
  • MuJoCo pilot where Jev, Claude Haiku and reactive rules choose pick-and-place actions; Jev matched the rules exactly.

    ProjectresearchPython