jev-system-one-rlcd
Local reproduction of Jev-style inference that reads option-token logits from a small LLM in one pass, with an honest failure log.
Overview
This repository reproduces the inference mechanism behind Jev-style decision models on a laptop, while being explicit that it does not reproduce Jev's weights or its full RLCD (Reinforcement Learning for Calibrated Decisions) stack. A bounded decision is phrased as a multiple-choice prompt with single-token letter labels; one prefill pass yields the last-position logits, and a softmax restricted to the option-letter tokens gives a real distribution over allowed answers. The model never generates text, mirroring SGLang's /v1/score approach.
The author also documents a first attempt that deleted the pretrained lm_head and attached untrained linear heads, which produced noise, and keeps it for comparison. On an M1 laptop, scoring with SmolLM-135M took about 60 ms versus about 1.6 s to generate JSON, roughly 26 times faster, but both 135M models mis-routed a test ticket at near-chance confidence. The README is clear that the mechanism is fast and structurally safe while accuracy needs a larger model and calibration. An RLCD training loop with proper scoring rules is included.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which of four teams should handle this support ticket? (letter-labelled options) | — |
How it works
- Format options with single-token letter labels
- Run one forward pass and take last-position logits
- Softmax over the option-letter tokens only
- Optionally train calibration with the RLCD loop
Strengths
- Invalid outputs are impossible by construction
- README reports ~19-26x speedup over generation on small models
- Candid account of what failed and why
Caveats
- 135M models route at near-chance accuracy
- Does not reproduce Jev weights or full calibration
- Research and teaching code
Getting started
pip install -r requirements.txt && python3 jev_scoring.pyAt a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- PyTorch / transformers
- Has code
- yes
Use cases and integrations
How curated lists describe it
Jev System One AI & RLCD Reproduction Engine: Sub-70ms Calibrated Decisions…
wh000wh000/awesome-jev-live
Frequently asked questions
Does this reproduce Jev?
Only the inference mechanism of reading option probabilities; not Jev's weights or its full calibration training.
How fast is it?
About 60 ms per decision on an M1 laptop with a 135M model, versus over a second for JSON generation.
Is it accurate?
Not at 135M parameters; the README notes accuracy needs a larger model and calibration.
Related projects
Exact-target audit finding Jev's Choice probabilities are a step function on stated base rates while per-option Noul tracks them closely.
ProjectresearchPythonResearch replicating Jev-style typed decisions with open LLMs via logprobs; Qwen3.5-9B reached 94% with good calibration in ~1 s.
ProjectresearchPythonOpen decision models that derive calibrated confidence from recurrent looped-layer dynamics instead of RL, with early exit and a Jev API.
ProjectresearchPythonLaptop-scale reimplementation of RL for Calibrated Decisions: parallel constrained decoding trained with GRPO and a Brier reward.
ProjectresearchPython- AnyJev★ 1,255
Nokia's training-free library that turns open LLMs into Jev-style deciders robust to option order, plus self-distilled Tacit models.
ModelresearchPython Jev-style replica: frozen Qwen3-1.7B plus a ~0.8M-parameter trained head returns calibrated Choice, Boolean and Score distributions.
ServiceresearchPython