Benchmarks and comparisons with decision models

Craftax survival agent where Jev picks one Choice per step from code-listed actions, with optional LLM goals, plus ablations.

Independent 52-page Chinese report on Jev that checks vendor claims and reproduces 50 Chinese tests, with traceable data.

Compares Jev's zero-shot reranking with specialist rerankers on MS MARCO passage and document benchmarks and a TREC WSJ transfer test.

Reproducible evaluation harness and field note for the Jev Ultrafast research browser, with QC-graded baseline and stress suites.

Short report comparing Jev and many other fast decision classifiers with an LLM on grading section relevance in technical documents.

Small Jev measurement bench with an append-only call log, labelled claims and a stated negative result.

Tetris where each reachable placement is an English sentence that Jev ranks, with controls showing a regex baseline does better.

Harness that runs Jev on all 19,646 TMMLU+ Traditional Chinese questions as four-way Choices, reporting accuracy and calibration.

Benchmark release evaluating Theory of Mind in five typed decision models versus seven generative baselines on nine ToM datasets.

Benchmark of BM25, embeddings, rerankers and Jev for picking the right MCP tool among 525, plus an experimental Jev search engine.

Research code evaluating Jev as an independent per-tool relevance judge for building task-level tool spaces on TGB, BFCL and GTA.

Point-in-time O'Neil momentum backtest where Jev picks entries and judges exits, with a jackknife showing no resolvable edge over rules.

Decision Lab: sixty finance demos asking Jev, OpenAI Decisions and twelve other decision models the same questions, graded.

RPG experiment pitting Jev against OpenAI's Decisions API as NPCs, comparing wins, per-move accuracy, calibration and cost.

Learning project benchmarking zero-shot Jev and GPT models against scikit-learn classifiers and ModernBERT on five datasets.

Benchmark harness pitting Jev against Claude Sonnet 5 and Haiku 4.5 on Banking77 intent sorting with a confidence gate.

Evaluates Jev as a fixed policy in Gymnasium and Atari games, picking a legal action each step via one Choice question.

C/C++ compiler driver where Jev decides whether to inline each discretionary call site, compared against Clang -Oz.

Research code showing that short, natural-looking context additions can flip decision models like Jev to a chosen wrong answer.

Benchmark of Jev against low-cost LLMs on 130,000 classification decisions across five datasets for accuracy, cost and latency.

Experiment testing whether a local Qwen read via label log-probabilities can replace Jev on synthetic AIOps alert decisions.

Open Jev-class decision models on Qwen3.5 with candidate-logit readout; the 4B model reports 80.1% on JevBench's 231 items.

Local Jev-style decision model on Qwen3.5-4B with four stress tests that show where one-pass decisions break.

LogitTrail (formerly Mini Jev): local workbench that reads typed Choice, Noul and Score decisions from a frozen LLM's logits.

Model Kombat: a 12,000-neuron fly connectome fights Jev in mk.js, with Jev picking one of seven moves about five times a second.

Reproduction and stress test of a paper asking whether Jev-style probabilities stay calibrated after routing on selected examples.

Open System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.

Independent reproduction of Jev-style decision-model post-training on MiniCPM5-2B with LoRA, a sealed holdout and stated limits.

Preliminary SHADE-Arena study of Jev as a monitor and per-action gate for covert agent sabotage, compared with Gemini 2.5.

Open Jev replica on Gemma 4 E2B and Gemma 3 270M served on Modal, with eight demos and a TypeSafe eval head-to-head.