open_system_one
Open System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.
Overview
Open System-1 is an Apache-2.0 research project building an open alternative to Jev. A System-1 model receives a context, a question and options and returns a calibrated probability for each option without writing text, supporting choice, bool and score questions. The project documents a series of experiments, and the current best model, exp9a with 4.9B parameters, was trained on a single Colab A100 in 2.4 hours.
On held-out benchmarks the README reports 73.3 on Banking77 (Jev 87.0, Laya 42.5), 84.2 on AG News, 63.0 on DAIR Emotion (above both), and 97.9 / 91.7 / 48.6 on JevBench easy, standard and hard public halves, noting the Jev and Laya figures come from a leaderboard on full tiers so comparisons are indicative. Across experiments, answers no longer change when options are reordered, and an answer-or-escalate utility metric improved. It suits researchers following open reproductions step by step.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which team should handle this ticket? | — |
| Yes / no | Does this claim hold, given the policy? | yes · no |
| Score | How urgent is this, on a 1-5 scale? | 1 · 2 · 3 · 4 · 5 |
How it works
- Encode instructions and context as a shared prefix.
- Score each option for the question.
- Return a calibrated distribution.
- Answer or escalate based on confidence.
Strengths
- Reproducible experiment log with honest comparisons.
- Order-invariant answers in exp9a.
- Low training cost (single A100, 2.4 h).
Caveats
- Trails Jev on Banking77, AG News and JevBench hard.
- Research status; comparisons are indicative.
At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- PyTorch
- Has code
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
Creating system one model just like jev. with different experimentation.
hellogumbo/awesome-jev
Frequently asked questions
What is the best model?
exp9a, a 4.9B-parameter model trained on one A100 in 2.4 hours.
Does it beat Jev?
Only on DAIR Emotion among the reported benchmarks.
Is it order-sensitive?
The README reports 0% answer changes when options are reordered for exp9a.
Related projects
Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPythonLightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.
ProjectresearchPythonInternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.
ModelresearchPythonVerdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.
ModelresearchPythonReproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.
ModelresearchPython- Verdict-open-jev★ 110
OpenJev: 151M ModernBERT decision model with RLCD-style calibration, a JevBench audit and an in-browser WebGPU playground.
BenchmarkresearchPython