lev
Open System One decision model (LoRA on Qwen3.5-4B) that reads typed answers from logits in one pass, with its benchmark harness.
Overview
lev is an open System One decision model and the harness that measures it. Given a state such as text, a ticket, an email or JSON, and a set of yes/no, choice and score questions, it answers all of them in one forward pass by reading each answer from logits it has already computed, returning calibrated probabilities over exactly the options supplied with no generated tokens. The model is a LoRA adapter on Qwen3.5-4B, and it speaks TypeSafe's /v1/systemone protocol so any TypeSafe client works by changing the base URL.
The README reports 68.9% across all 13 S1Bench subsets, level with reflex-4b on the six subsets the public board completed and behind only Jev and three open models of 26B to 35B. Per-subset results are shown against Jev and a majority-label baseline; for example lev beats Jev on MultiNLI (0.890 vs 0.836) but trails on PAWS (0.776 vs 0.900) and VitaminC claim verification. The README stresses that the structural guarantee is that answers stay within your options, not that they are correct. It includes a decision log, findings, a Snake demo and training instructions, with a write-up on the Interfaze blog.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Any yes/no question (e.g. is this claim supported?) | — |
| Choice | Which intent or label applies (e.g. 18 MASSIVE scenarios)? | — |
| Score | Where does the state fall on 2-10 ordered levels? | — |
How it works
- Install lev with the serve extra
- Serve /v1/systemone on a GPU
- Point a TypeSafe client at the server
- Receive calibrated probabilities over your options
- Use the harness to measure on S1Bench subsets
Strengths
- Open weights with a reproducible harness
- Answers cannot fall outside your options
- Drop-in for TypeSafe clients
- Published per-subset comparison with Jev
Caveats
- Trails Jev on most subsets
- Needs a CUDA GPU for real-time use
- Structural guarantee is not correctness
Getting started
pip install "lev[serve] @ git+https://github.com/Abhinavexists/lev#subdirectory=packages/lev"At a glance
- Type
- Model
- Status
- beta
- Language
- Python
- Runtime
- PEFT LoRA on Qwen3.5-4B with HTTP server
- Has code
- yes
- Has tests
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
An open System One decision model.
wh000wh000/awesome-jev-liveAn open System One decision model.
hellogumbo/awesome-jevA LoRA adapter on Qwen3.5-4B that answers typed questions from logits it has already computed, served over `/v1/systemone` so existing clients work against it, with the harness that measures it.
MrJev/awesome-jev
Frequently asked questions
How does lev compare with Jev?
68.9% across 13 S1Bench subsets; it trails Jev on most subsets but beats it on MultiNLI, per the README.
Can I use my TypeSafe client?
Yes, change the base URL to a lev server.
Can it return a label I did not list?
No, the answer space is the option set you send.
Related projects
- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython Reads Jev-style choice, yes/no and score probabilities from masked diffusion language models with shared Yes/No weights.
ProjectresearchPythonOpen Jev-class decision models on Qwen3.5 with candidate-logit readout; the 4B model reports 80.1% on JevBench's 231 items.
ProjectresearchPythonOpen Jev replica on Gemma 4 E2B and Gemma 3 270M served on Modal, with eight demos and a TypeSafe eval head-to-head.
ProjectresearchPythonSmall experiment: reordering enum options changed a local Jev clone's answers on its old default, while hosted Jev's did not.
DemoresearchPython- JevK5★ 150
Apache-2.0 open-weight Jev alternative that answers yes/no, choice and score questions with per-option probabilities in one pass.
LibrarybetaPython