Qwev
Training-free runtime turning local Qwen3/Qwen3.5 checkpoints into a Jev-style Noul/Choice/Score service via candidate-token logits.
Overview
Qwev (Qwen + Jev) applies Jev's state-and-questions interface to existing dense Qwen3 and Qwen3.5 checkpoints without fine-tuning. The runtime encodes the shared state once, evaluates each question in an isolated branch that can see the state and its own prompt but not other questions, and reads the decision directly from the candidate-token logits, so nothing is generated or parsed.
Requests use the familiar format: a state plus questions with type choice, noul or score and criteria. An inspection command checks a local model's config before loading weights, and a shared execution mode is used for the main path. On the included 27-question fixture with Qwen3.5-9B, the README reports 0.500 s for the full request versus 13.554 s for the same model generating a JSON answer (27.1x faster). It is an independent implementation built on public checkpoints and does not use Jev weights, data or methods.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which team should own the primary issue? Billing takes priority. | returns · shipping · billing |
| Yes / no | Was a duplicate charge reported? | — |
How it works
- Download a dense Qwen checkpoint and inspect it with --inspect-model
- Send a request with a state and typed questions
- Prefill the state once and run isolated question branches
- Return typed answers with distributions over the declared answer space
Strengths
- No training required
- 27.1x faster than generated JSON on the included fixture per the README
- Question isolation prevents cross-question leakage
- Runs entirely locally
Caveats
- Zero-shot logits from a base chat model are not necessarily calibrated
- Needs a GPU for 9B-class models
- Only dense Qwen3/3.5 checkpoints supported
Getting started
pip install -e . then qwev --model-path /models/Qwen3.5-9B --request examples/basic.json --device cuda:0 --dtype bfloat16 --mode sharedAt a glance
- Type
- Service
- Status
- prototype
- Language
- Python
- Runtime
- Transformers / PyTorch CLI and service
- Has code
- yes
Use cases and integrations
How curated lists describe it
Local inference: turns dense Qwen3 and Qwen3.5 checkpoints into a training-free Jev-style `Noul`, `Choice`, and `Score` service that shares one state prefill across isolated questions and, on its included 27-question Qwen3.5-9B/A100 fixture, reports 0.500 s versus 13.554 s for generated JSON.
yibie/awesome-jev
Frequently asked questions
Does Qwev need fine-tuning?
No; it reads decisions from existing Qwen checkpoints' candidate-token logits.
How much faster is it than generating JSON?
On the included 27-question fixture the README reports 0.500 s versus 13.554 s.
Does it use Jev?
No; it adopts Jev's interface ideas only and needs no TypeSafe key.
Related projects
Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoTurns any chat model on SGLang, vLLM, transformers or MLX into a Jev-compatible service with one prefill and one label token per question.
ProjectbetaPythonSelf-hosted Jev-compatible System One server that sits in front of llama-server and reads option probabilities from any GGUF model.
ServicebetaPythonInference engine for Jev-style decision models using segmented forwards with prefix reuse, serving the Jev wire contract.
ProjectresearchPythonResearch preview: Python client and pinned adapters for running System One decision models on your own Modal GPUs.
LibraryresearchPython