Decision Models in llama.cpp

Hugging Face blog on llama.cpp's /v1/systemone endpoint serving decision models locally in Jev's wire format.

Serviceproduction

This summary is based on how curated lists describe the project; its README was not available to review.

Overview

This Hugging Face blog post by ggml-org describes decision-model support in llama.cpp. According to the list summaries, llama-server adds a /v1/systemone endpoint that takes a state, questions and candidate answers and returns the chosen option with a probability for each, in Jev's own wire format. It serves models such as Laya, Kev-4B, OpenJev and Clef, supports image state, and has a router mode that loads several models on demand, with official tests from 144M up to 27B models.

For developers, this makes Jev-compatible typed decisions available from the same local runtime many already use for LLMs, with no hosted dependency.

What the decision model decides

Type	Question	Answers
Choice	Typed questions with candidate answers posed to a local model	—

How it works

Run llama-server with a decision model
POST state and questions to /v1/systemone
Receive the chosen option and per-option probabilities

Strengths

Local, Jev-compatible endpoint
Image state support
Router mode for multiple models

Caveats

List-only entry; blog not reviewed
Model quality varies by checkpoint

Getting started

See https://huggingface.co/blog/ggml-org/decision-models-in-llamacpp.

At a glance

Type: Service
Status: production
Language: C++
Runtime: llama-server
Has code: yes
Has tests: yes

Use cases and integrations

Infra / SDKs / Integrations537

How curated lists describe it

Local inference: llama.cpp adds a `/v1/systemone` endpoint that takes a state, questions and candidate answers and returns the chosen option with a probability for each, in Jev's own wire format, with official tests from 144M up to 27B models.
yibie/awesome-jev
`llama-server` serves decision models such as Laya, Kev-4B, OpenJev, and Clef at `/v1/systemone`, with image state and a router mode that loads several models on demand
AnotiaWang/awesome-decision-models

Frequently asked questions

Does llama.cpp speak Jev's API?

According to the summaries, llama-server exposes /v1/systemone in Jev's wire format.

Which models can it serve?

Models such as Laya, Kev-4B, OpenJev and Clef, from 144M to 27B in official tests.

Does it support images?

Yes, image state is supported per the summary.

Related projects

vLLM Jev★ 524
Native vLLM serving for Jev-style decision models on Linux and Apple Silicon, including image and video questions for Valen.
ServicebetaPython
openjev-multimodal
Local Jev-compatible API on a Mac: Qwen via llama.cpp/Metal reads one output token per question for text and image decisions.
ModelbetaPython
Lichen★ 58
Drop-in local replacement for Jev that serves /v1/systemone from open weights, including questions about images.
ServicebetaPython
Jev Local
Local /v1/systemone server in Japanese docs with LFM, Sarashina and fine-tuned ModernBERT-Ja backends, including image input.
ServicebetaPython
NeoHorse-Jev-4B
Hugging Face card for NeoHorse-Jev-4B: Apache-2.0 4B decision model for text or image-plus-text, served at /v1/systemone.
ModelbetaPython
Standard One
Standard Thinking's Apache-2.0 3B and 8B decision models on Ministral 3 that score bounded answers via /v1/systemone, with vision.
ServicebetaPython