llm2jev
Turns any chat model on SGLang, vLLM, transformers or MLX into a Jev-compatible service with one prefill and one label token per question.
Overview
llm2jev (tic-top), formerly AnyJev, turns any chat model into a Jev-compatible probability decision service without model or engine changes. For each question it puts the state, the question and all options into one prompt and reads the log-probabilities of the option labels at a single position: one forward pass per question regardless of option count, up to 255, with no generation or sampling. Because all options appear together they compete directly, so a 'none of the above' option can depend on the others.
It uses the standard chat template with thinking off, so instruct models work zero-shot and fine-tuned checkpoints use the same prompt. Every question in a request starts with byte-identical text, so the shared prefix is warmed once and the engine's prefix cache (SGLang radix or vLLM APC) serves the rest. Images, video and audio can be part of the state on supported backends. The README gives detailed setup for each backend, including the vLLM flags required for log-probabilities. It is an independent implementation of the documented System One wire format, with Chinese documentation too.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which option label fits (up to 255 options)? | — |
| Yes / no | Any yes/no question | — |
How it works
- Launch a chat model on SGLang, vLLM, transformers or MLX
- Start llm2jev pointing at the backend
- Each question becomes one prompt with all options
- Read option-label log-probabilities after one prefill
- Return Jev-format probabilities
Strengths
- Works with any chat model and major engines
- Options compete in one prompt
- Prefix caching for efficiency
- Multimodal state support
Caveats
- Calibration depends on the chat model
- vLLM needs specific flags
- Transformers backend is slow
Getting started
pip install llm2jev, launch SGLang with your model, then llm2jev --model Qwen/Qwen3.5-2B --backend sglang --url http://127.0.0.1:30000At a glance
- Type
- Project
- Status
- beta
- Language
- Python
- Runtime
- server and library over SGLang, vLLM, transformers or MLX
- Has code
- yes
Use cases and integrations
How curated lists describe it
Any chat model, any engine (SGLang, vLLM, transformers) as a Jev-compatible probability decision service: one prefill, one label token.
hellogumbo/awesome-jev
Frequently asked questions
How does it score options?
It reads log-probabilities of option labels at one position after a single prefill.
Which engines are supported?
SGLang, vLLM, transformers and MLX.
Can it use images or audio?
Yes, on SGLang, vLLM and transformers backends with multimodal models.
Related projects
Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoSelf-hosted Jev-compatible System One server that sits in front of llama-server and reads option probabilities from any GGUF model.
ServicebetaPythonInference engine for Jev-style decision models using segmented forwards with prefix reuse, serving the Jev wire contract.
ProjectresearchPythonResearch preview: Python client and pinned adapters for running System One decision models on your own Modal GPUs.
LibraryresearchPython- simple-jev★ 602
Featherless AI server that turns compatible open Hugging Face models into a /v1/systemone classifier by reading next-token logits.
ServicebetaPython