Verdict-open-jev
OpenJev: 151M ModernBERT decision model with RLCD-style calibration, a JevBench audit and an in-browser WebGPU playground.
Overview
OpenJev (Verdict) is an open-source decision model inspired by Jev and Reinforcement Learning for Calibrated Decisions (RLCD), built on a 151M ModernBERT backbone. It takes unstructured context and evaluates multiple typed questions in a single non-autoregressive pass, returning choices, ordinal scores and binary probabilities with calibrated confidence in under 35 ms, per the README. Weights are on Hugging Face and a WebGPU playground runs it in the browser.
A recent inference-only update fixed three defects: calibrators not loading and only applying to five-option questions, candidate labels not phrased as NLI hypotheses, and an oversized context budget. On the 231 public JevBench tasks, easy-tier accuracy went from 85.4 to 87.5 percent, standard from 62.5 to 69.4 percent, hard stayed at 36.9 percent, and hard-tier ECE fell from 0.298 to 0.118. The weak hard-tier result is reported openly.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which department or team owns this support ticket? | — |
| Yes / no | Does this transaction satisfy our fraud threshold? | yes · no |
How it works
- Load the 151M model locally or in the browser.
- Send context and typed questions.
- Receive calibrated choices, scores and probabilities in one pass.
- Threshold on confidence.
Strengths
- Small and fast; runs in the browser via WebGPU.
- Calibration improved substantially in the latest engine.
- Transparent JevBench audit.
Caveats
- Hard-tier accuracy is low (36.9 percent).
- Trained on short contexts.
At a glance
- Type
- Benchmark
- Status
- research
- Language
- Python
- Runtime
- PyTorch, WebGPU browser playground
- Has code
- yes
- Stars
- 110
Use cases and integrations
How curated lists describe it
Non-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-browser WebGPU playground
wh000wh000/awesome-jev-liveNon-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-browser WebGPU playground.
hellogumbo/awesome-jev
Frequently asked questions
How big is OpenJev?
151M parameters on ModernBERT.
How does it do on JevBench?
87.5 percent easy, 69.4 percent standard and 36.9 percent hard tier after the latest fixes.
Can I try it in the browser?
Yes, there is a WebGPU playground.
Related projects
Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPythonOpen System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.
ProjectresearchPythonLightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.
ProjectresearchPythonInternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.
ModelresearchPythonVerdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.
ModelresearchPythonReproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.
ModelresearchPython