LLM2Jev
Zero-training toolkit that turns local LLMs and multimodal models into Jev-style decision services via prefill-only scoring.
Overview
LLM2Jev turns an existing LLM or multimodal LLM into a Jev-style decision model with no training. It gets decision probabilities from prefill alone across SGLang, Transformers and MLX backends, handling classification, scoring and action selection from text and images. With SGLang, chat and the /v1/systemone decision endpoint share one inference engine and loaded model, and a Python client extending the OpenAI SDK offers systemone.create() alongside chat.
Its news log records a v0.1.0 release, multimodal inputs, an MLX backend with candidate batching and bounded prefix reuse, staged candidate submission to reuse SGLang's radix cache on cold requests, Snake and web demos, and a visual Sokoban evaluation. A JevBench evaluation on 231 public items reports P50 latency under a tenth of the official Jev result, with accuracy documented separately. It is independent and not affiliated with TypeSafe, with documentation in English and Chinese and a project website.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which option or action fits the text or image? | — |
| Score | Where does the input fall on the scale? | — |
| Yes / no | Is this statement true? | — |
How it works
- Serve an existing model with SGLang, Transformers or MLX
- Send typed questions via /v1/systemone or the Python client
- Prefill scores candidate options; prefix cache is reused
- Probabilities are returned in Jev's shape
Strengths
- No training needed
- Shares one model between chat and decisions
- Multimodal support
- Fast reported latency
Caveats
- Calibration depends on the base model
- Accuracy versus Jev documented separately
- Early release (v0.1.0)
Getting started
See docs/en/usage.md for installing and serving with SGLang, Transformers or MLX.At a glance
- Type
- Service
- Status
- beta
- Language
- Python
- Runtime
- SGLang, Transformers or MLX with HTTP service
- Has code
- yes
Use cases and integrations
How curated lists describe it
Turn local language models into Jev-style structured decision models.
wh000wh000/awesome-jev-liveAdapt local language models into Jev-compatible structured decision engines with Choice, Score, and Noul outputs powered by prefill-only binary inference.
hellogumbo/awesome-jevAdapts a local language model into a Jev-style decision engine, answering runtime-defined Choice, Score and Noul questions through SGLang.
MrJev/awesome-jev
Frequently asked questions
Do I need to fine-tune?
No, it uses existing model weights with prefill-only scoring.
Can it handle images?
Yes, multimodal inputs are supported on SGLang, Transformers and the HTTP API.
How is it different from llm2jev by tic-top?
They are separate projects with similar goals.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonReads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
ProjectresearchPython