Local runtimes and serving with decision models

TurinTech GGUF builds of Clef, Clef-flash, pplx-decider and Kev for stock llama.cpp at /v1/systemone, at about 27-30% of full size.

vLLM Semantic Router's Apache-2.0 decision models (0.6B to 27B) answering up to 64 typed questions per input in one forward pass.

SGLang's native /v1/decisions and /v1/systemone endpoints that read option probabilities from a validated chat model's next-token scores.

Ollama blog post: Ollama serves the /v1/systemone decision endpoint locally since 0.35, starting with nimble and tev1 models.

James Cruce's write-up on choosing local LAYA over hosted Jev: the 421M open model beat a home-made router on a frozen replay.

Unsloth desktop app docs page: serves LAYA and Clef decision models locally through a TypeSafe-compatible /v1/systemone API.

Hacker News thread about Jeyzma, pitched as a Jev-style decision model run in the browser via Go, TinyGo and llama.cpp.

Japanese X post about running a Jev-style decision model locally, noting speed is still improvable.

Show HN thread for AT0M, a Jev-like System One decision model packaged as a single Rust binary.