TypeAR
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
Overview
TypeLLM (listed here as TypeAR) adds type-safe generation to existing autoregressive LLMs without changing their weights. Inspired by Jev, it produces schema-guaranteed outputs from JSON Schema while keeping models' native thinking and free-form generation. It is built on SGLang and supports strings, integers, numbers, booleans, enum choices, objects and arrays.
Features added over time include permutation averaging to improve choice distributions, dependency graphs with prefix reuse, conditional fields that can depend on another answer's confidence, optional thinking with per-field budgets, and image input. The README reports JevBench results of 195 of 231 without thinking and 228 of 231 with thinking. A hosted TypeLLM API is also offered. It is relevant to anyone who wants Jev-style typed decisions from open LLMs they already serve.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Enum choice field defined in a JSON Schema | — |
| Other | Conditional field answered only when another answer's confidence is at least a threshold | — |
How it works
- Define output fields in JSON Schema.
- TypeLLM constrains decoding so each field stays in schema.
- Choice distributions can be averaged over option permutations.
- Dependent and conditional fields are answered in order with prefix reuse.
Strengths
- No out-of-schema outputs.
- Works with existing open LLMs without retraining.
- Reported JevBench scores of 195 and 228 of 231.
- Permutation averaging reduces option-order bias.
Caveats
- Requires serving models on SGLang or using the hosted API.
- Generation is still autoregressive, so latency differs from single-pass decision models.
At a glance
- Type
- Project
- Status
- beta
- Language
- Python
- Runtime
- SGLang server, hosted API
- Has code
- yes
- Stars
- 946
Use cases and integrations
How curated lists describe it
TypeLLM: LLMs with type-safe generation
wh000wh000/awesome-jev-liveType-safe one-decision-per-token decoding engine for autoregressive LLMs, inspired by Jev.
hellogumbo/awesome-jev
Frequently asked questions
How does TypeLLM relate to Jev?
It is inspired by Jev and brings typed, schema-guaranteed outputs to ordinary autoregressive LLMs.
How did it score on JevBench?
The README reports 195 of 231 without thinking and 228 of 231 with thinking.
Do I have to self-host?
No, a hosted TypeLLM API is available, or you can run it on SGLang.
Related projects
Reads Jev-style choice, yes/no and score probabilities from masked diffusion language models with shared Yes/No weights.
ProjectresearchPythonOpen Jev-class decision models on Qwen3.5 with candidate-logit readout; the 4B model reports 80.1% on JevBench's 231 items.
ProjectresearchPythonOpen Jev replica on Gemma 4 E2B and Gemma 3 270M served on Modal, with eight demos and a TypeSafe eval head-to-head.
ProjectresearchPythonSmall experiment: reordering enum options changed a local Jev clone's answers on its old default, while hosted Jev's did not.
DemoresearchPython- JevK5★ 150
Apache-2.0 open-weight Jev alternative that answers yes/no, choice and score questions with per-option probabilities in one pass.
LibrarybetaPython Open System One decision model (LoRA on Qwen3.5-4B) that reads typed answers from logits in one pass, with its benchmark harness.
ModelbetaPython