Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

dohnuts.cpp

Native C++ (llama.cpp-based) CPU inference for Dohnuts and other small decision models behind a /v1/systemone server.

Modelbeta
View on GitHub ↗

Overview

dohnuts.cpp provides native C++ inference for Dohnuts, built on llama.cpp. It runs the released Qwen3.5-0.8B base with the merged Dohnuts LoRA, scores candidate markers with a scalar decision head and returns probabilities from one forward pass, with no GPU and no text generation. An optional mmproj file enables image decisions through the base model's frozen vision tower.

The binary serves a /v1/systemone endpoint and a /predict endpoint that accepts single or batched requests. Through profiles it also runs the larger Linnaeus-0.1.0-2B and other decision models such as decider, thisthat and kev. The README recommends llama-server for models it already supports, and says this project will stay focused on the Dohnuts family and early support for small side models. Documentation is also available in Chinese.

What the decision model decides

TypeQuestionAnswers
ChoiceWhich team should handle this?billing · technical support · sales
Yes / noIs a refund requested?yes · no

How it works

  1. Convert or download the GGUF model and head
  2. Start dohnuts-cli --server
  3. POST state and questions to /v1/systemone
  4. Receive probabilities and confidence per question

Strengths

  • Runs on CPU with no GPU
  • Jev-compatible endpoint
  • Supports several small decision models via profiles

Caveats

  • Narrow scope; llama-server preferred where supported
  • Inherits Dohnuts weight licence restrictions

Getting started

build/dohnuts-cli --server --port 8080 --model work/dohnuts-Q8_0.gguf --head work/head.f32 --metadata models/dohnuts-0.1.0/dohnuts.json.

At a glance

Type
Model
Status
beta
Language
C++
Runtime
CLI / HTTP server
Has code
yes
Stars
11

Use cases and integrations

How curated lists describe it

  • The same decisions, on CPU. System One model that can run on your Personal Computer.

    wh000wh000/awesome-jev-live

  • The same decisions, on CPU. System One model that can run on your Personal Computer.

    hellogumbo/awesome-jev

  • Native C++ inference for Dohnuts on the released Qwen3.5-0.8B base, so the same decisions run on a CPU.

    MrJev/awesome-jev

Frequently asked questions

Does it need a GPU?

No, it runs on CPU.

Which models does it run?

Dohnuts, Linnaeus-0.1.0-2B and others such as decider, thisthat and kev via profiles.

Should I use it instead of llama-server?

The README recommends llama-server for models it already supports.

Related projects

  • decidealot

    Docker service exposing local Laya, Von, a CLM head or hosted Jev through one TypeSafe-compatible HTTP and MCP endpoint.

    MCP serverbetaPython
  • edgejev

    Converts open Jev reproductions (Laya, Kev, NanoJev, PlayJev) to INT8 ONNX for offline CPU decisions at about 15.6 ms per question.

    ServicebetaPython
  • go-system-one

    Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.

    ProjectbetaGo
  • gojev

    Go harness for System One models: hosted Jev, open Kev in-process via llama.cpp, and Cloudflare Clef, with typed helpers and a CLI.

    ServicebetaGo
  • hearim

    Go gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.

    ServicebetaGo
  • intern-decision-mlx

    Runs Intern-Decision-0.8B on Apple Silicon behind a local /v1/systemone endpoint that accepts screenshots for agent decisions.

    ServicebetaPython