system-one
Python library that turns an open LLM into a Jev-style classifier via constrained single-token choice indexes in one forward pass.
Overview
System One by sgoedecke turns any Hugging Face chat LLM into a System One style classifier like Jev. Given a state and a set of Choice questions in TypeSafe SDK form, it uses the model's chat template, prefills choice_index:, and constrains the logits to tokens that match a choice index, so the most likely index is the answer and its probabilities come from a single forward pass. A cache_prefix option prefills the shared state once and evaluates all question suffixes, which the author recommends for more than three questions.
The author reimplemented the Doom and Wikipedia-race demos from the Jev launch post with Qwen3-8B. The README reports that the same model with ordinary tool calls was about 3.5 times slower between Doom actions (600 ms versus 172 ms median) and tended to issue one input per turn. The Wikipedia race used labels instead of indexes because pages had over a thousand links. The author notes the code is vibe-coded.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which team should handle this? | returns · shipping · billing |
| Choice | What is the tone? | calm · frustrated · angry |
How it works
- Load an open model with SystemOne.from_pretrained.
- Pass state and TypeSafe-style Choice questions.
- Prefill the answer prefix and constrain logits to valid choice indexes.
- Read the chosen option and probabilities per question.
Strengths
- Works with any open chat model, no hosted API needed.
- Shared-prefix caching batches many questions cheaply.
- Compatible with TypeSafe SDK question objects.
Caveats
- Author describes the code as vibe-coded.
- No licence stated in the README.
- Uncalibrated raw model probabilities.
Getting started
pip install -e . then SystemOne.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct").system_one(state=..., questions=...).At a glance
- Type
- Project
- Status
- prototype
- Language
- Python
- Runtime
- library (Hugging Face transformers)
- Has code
- yes
Use cases and integrations
How curated lists describe it
Batched single-token choice inference for open language models, compatible with TypeSafe.
hellogumbo/awesome-jev
Frequently asked questions
How does it get a decision in one forward pass?
It prefills choice_index: and restricts the next token to valid indexes, taking the most likely one as the answer.
Does it need a TypeSafe key?
No, it runs an open model locally and only reuses the TypeSafe SDK's Choice type.
How did it compare with tool calling?
In the Doom demo the README reports 172 ms median between actions versus 600 ms with ordinary tool calls on the same model.
Related projects
Local Snake game where code computes facts for each legal move and a local Qwen3.5 picks one in a single Jev-style Choice pass.
AppprototypePythonJapanese article testing whether LLMs can replicate Jev, via a logit shortcut on Gemma and the Mario harness.
Articlediscussion- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonOpen Jev-style decision model built on Gemma 3 270M with a scoring head, scoring all options in one forward pass on CPU.
ProjectresearchPython