verdict
Self-hosted Jev-compatible System One server that sits in front of llama-server and reads option probabilities from any GGUF model.
Overview
verdict is an open-source, self-hosted alternative to Jev's System One API. A small Python server sits in front of an existing llama-server or llama-swap endpoint and serves the Jev wire protocol, so an existing Jev client only changes its base URL. For each question it runs one forward pass, reads the probability the model puts on each declared option label, renormalizes, and returns a distribution plus confidence signals, including how much mass landed on the options at all.
No tokens are sampled, so there is no seed. The README measures determinism honestly: on a quiet server, eight models reproduced scores exactly, but under load 5 of 16 prompts differed by up to 0.032 because llama.cpp batching changes reduction order. It publishes JevBench results for several GGUFs; the best listed, jev-omni Q8_0, scored 204 of 231 with hard-tier ECE 0.093 at about 0.52 s p50.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which of these links opens the story's comments? | 229 comments · Hacker News · hide · past |
How it works
- Run llama-server with any GGUF model.
- Start verdict in front of it.
- Point a Jev client's base URL at verdict.
- Read renormalized option distributions and option mass.
Strengths
- Any local GGUF model can serve Jev-style decisions.
- No sampling, so no seed-dependent randomness.
- Published JevBench numbers per model and noise floor.
Caveats
- Results vary slightly under concurrent load.
- Accuracy depends on the chosen model.
At a glance
- Type
- Service
- Status
- beta
- Language
- Python
- Runtime
- HTTP server in front of llama.cpp
- Has code
- yes
Use cases and integrations
How curated lists describe it
Turn any llama-server into a jev system one endpoint.
hellogumbo/awesome-jev
Frequently asked questions
Do I need to change my Jev client?
Only the base URL.
Is it deterministic?
Exactly on a quiet server; under load some prompts differ by up to 0.032.
Which model scored best?
jev-omni Q8_0, 204 of 231 on public JevBench in the README.
Related projects
Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoTurns any chat model on SGLang, vLLM, transformers or MLX into a Jev-compatible service with one prefill and one label token per question.
ProjectbetaPythonInference engine for Jev-style decision models using segmented forwards with prefix reuse, serving the Jev wire contract.
ProjectresearchPythonResearch preview: Python client and pinned adapters for running System One decision models on your own Modal GPUs.
LibraryresearchPython- simple-jev★ 602
Featherless AI server that turns compatible open Hugging Face models into a /v1/systemone classifier by reading next-token logits.
ServicebetaPython