SNAP
Local, deterministic Rust decision engine reading single-letter answer logits from one llama.cpp pass, wire-compatible with Jev.
Overview
snap (Single-pass Neural Answer Probabilities) is a decision engine for LLM pipelines: you send your data and every question you have about it, and get back typed answers each with a full probability distribution from a single pass over a local model, with no generated text to parse, retry or clean up. Each question becomes a prompt whose answer is a single letter A to Z, one per option; snap reads the logits at that one position, merges tokens spelling the same letter and applies a softmax. All questions in a request share one batched llama_decode call.
POST /v1/systemone speaks the TypeSafe/Jev wire format, so a Jev client works unchanged against a local snap serve, and optional extensions add status, confidence, coverage, level and an allow_abstain option that lets the model say it cannot tell. Intended uses include ticket routing, checking an agent's tool call before it runs, incident severity, listing moderation, search relevance, faithfulness checks and lead qualification, each shown on the project site with real requests. It is MIT-licensed Rust and is not affiliated with TypeSafe.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Route this ticket: billing or tech? | billing · tech |
| Yes / no | SLA breach likely? | — |
| Score | What is the impact? | low · mid · high |
How it works
- Run snap serve with a local model
- POST state and questions in Jev's format
- Read letter-position logits in one batched pass
- Return distributions with confidence and coverage; abstain if allowed
Strengths
- Deterministic and local
- Drop-in for Jev clients
- Abstention and coverage signals for thresholds
Caveats
- Accuracy depends on the chosen open model
- Not affiliated with TypeSafe
Getting started
Download a release binary and run snap serve, then point a Jev client at its /v1/systemone endpoint.At a glance
- Type
- Demo
- Status
- beta
- Language
- Rust
- Runtime
- Local server on llama.cpp
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Typed decisions from unstructured state: one forward pass, zero generated text. Local, deterministic, Jev-compatible. Not affiliated with typesafe.ai.
hellogumbo/awesome-jevLocal, deterministic typed decisions from one forward pass of an open model, serving the Jev request shape.
MrJev/awesome-jev
Frequently asked questions
Can my Jev client use snap?
Yes, /v1/systemone speaks the same wire format.
How does it avoid generating text?
Each option gets a letter and snap reads the probability of each letter at one position.
Can it say it doesn't know?
Yes, with allow_abstain, and it reports coverage and confidence.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonReads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
ProjectresearchPython