Jeeves
PostHog's 9B Jev-like model that reasons before deciding, trained with SFT and CISPO, with a diffusion drafter for speed.
Overview
Jeeves, from PostHog, starts from the observation that Jev-like models give calibrated probabilities but at limited accuracy, so many pipelines fall back to a reasoning model. Jeeves trains a Jev-style classifier on Qwen3.5-9B with a LoRA and a pointer head to think before it decides, using supervised fine-tuning and CISPO reinforcement learning, and adds a block-4 diffusion drafter to speed up the reasoning chain. It is inspired by Kev.
It supports noul, choice and score questions in one request through a Jev-compatible API. The README reports 0.889 overall on held-out test data versus 0.822 for Kev-9B and 0.857 for Jev, and 0.935 versus 0.866 for Jev on JevBench's 231 public items, though Jev scores higher on a transfer set (0.800 vs 0.746). Latency is about 0.3 s per request without thinking and a 3.3 s median with it on one H100 in FP8. Training code and train, dev and test data are public, and it runs on CUDA or Apple Silicon.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Yes/no question | yes · no |
| Choice | Multiple-choice question | — |
| Score | Rating question | — |
How it works
- Receive a Jev-compatible request
- Optionally generate a reasoning chain (accelerated by the diffusion drafter)
- Pointer head selects answers with probabilities
- Return typed answers
Strengths
- README reports higher accuracy than Jev on held-out data and JevBench public
- Open training code and data
- Reasoning can be turned off for 0.3 s responses
Caveats
- Thinking mode is much slower (3.3 s median on H100)
- Lower than Jev on the transfer benchmark per the README
- Needs a capable GPU or Apple Silicon
At a glance
- Type
- Model
- Status
- research
- Language
- Python
- Runtime
- PyTorch (CUDA or MPS)
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
PostHog's 9B Jev-like model (Qwen3.5-9B with a LoRA and a pointer head) trained with CISPO to reason before it decides, with a diffusion drafter for speed; it reports 0.935 against 0.866 for Jev on JevBench's 231 public items. Training code and data are public.
AnotiaWang/awesome-decision-modelsJeeves – Reasoning improves Jev-like decision models.
wh000wh000/awesome-jev-liveA Jev-style classifier that reasons before it decides: Qwen3.5-9B with a LoRA and a pointer head, trained with SFT and CISPO, plus a diffusion drafter. From PostHog.
MrJev/awesome-jev
Frequently asked questions
How is Jeeves different from Jev?
It reasons before deciding, trading latency for accuracy, and is open.
How does it compare on benchmarks?
The README reports 0.935 vs 0.866 for Jev on JevBench public items, but lower on a transfer set.
Is it fast?
About 0.3 s without thinking and 3.3 s median with thinking on one H100 in FP8.
Related projects
- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython Reads Jev-style choice, yes/no and score probabilities from masked diffusion language models with shared Yes/No weights.
ProjectresearchPythonOpen Jev-class decision models on Qwen3.5 with candidate-logit readout; the 4B model reports 80.1% on JevBench's 231 items.
ProjectresearchPythonOpen Jev replica on Gemma 4 E2B and Gemma 3 270M served on Modal, with eight demos and a TypeSafe eval head-to-head.
ProjectresearchPythonSmall experiment: reordering enum options changed a local Jev clone's answers on its old default, while hosted Jev's did not.
DemoresearchPython- JevK5★ 150
Apache-2.0 open-weight Jev alternative that answers yes/no, choice and score questions with per-option probabilities in one pass.
LibrarybetaPython