Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

DuoMind

OpenAI-compatible local server pairing a small llama.cpp model for generation with Jev for the closed decisions.

Serviceprototype
View on GitHub ↗

Overview

DuoMind is a dual-system, OpenAI-compatible local LLM server. A small model such as Phi-4, Qwen, Gemma or Llama runs locally via llama.cpp as System 2 for generation, while Jev serves as System 1 for classification and decision-making, so generation stays on your machine and a 3B model is meant to behave like a larger one.

The README reports a benchmark on Qwen2.5-Coder-3B over 100 requests across five prompt types: with Jev enabled, about ten Jev decisions per request, average response time rose 36% (about 0.24 seconds of decision overhead), while completion length rose 60%, prompt tokens fell 15% and LLM generation speed rose 17%. The authors interpret longer completions as more complete answers; the README does not report a quality evaluation of the outputs.

What the decision model decides

TypeQuestionAnswers
ChoiceClosed decisions about the prompt that steer the local model (about ten per request)(inferred)—

How it works

  1. Client sends an OpenAI-compatible request
  2. Jev answers the closed decision questions
  3. Local llama.cpp model generates guided by those decisions
  4. Response returned through the OpenAI-compatible API

Strengths

  • Generation stays local and private
  • Drop-in OpenAI-compatible API
  • Measured effect on tokens and speed

Caveats

  • Adds latency per request
  • No output quality evaluation; longer answers are not necessarily better
  • Prompt text is sent to Jev for decisions

Getting started

See the README for setting up llama.cpp with a small model and configuring the Jev key.

At a glance

Type
Service
Status
prototype
Language
Python
Runtime
local OpenAI-compatible server (llama.cpp)
Has code
yes

Use cases and integrations

How curated lists describe it

  • OpenAI-compatible local server that pairs a small llama.cpp model for generation with a decision model for the closed questions, so generation stays on your machine.

    MrJev/awesome-jev

Frequently asked questions

What does Jev do in DuoMind?

It handles the closed classification and decision steps while a local model generates text.

Does it slow responses?

Yes, the README reports 36% higher average response time with Jev enabled.

Is generation local?

Yes, text generation runs on your machine via llama.cpp.

Related projects

  • go-system-one

    Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.

    ProjectbetaGo
  • hearim

    Go gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.

    ServicebetaGo
  • instinct-ai

    Open-source dual-brain runtime that answers typed decisions with a fast local engine or the Jev API before falling back to LLMs.

    ServicebetaPython, TypeScript
  • jev-switch

    Unofficial Rust gateway routing /v1/systemone requests to TypeSafe, Vercel, OpenRouter or local Laya via an editable DAG.

    ProxybetaRust
  • llm2jev

    Turns any chat model on SGLang, vLLM, transformers or MLX into a Jev-compatible service with one prefill and one label token per question.

    ProjectbetaPython
  • one-system

    Go gateway that puts local and hosted System One models behind one TypeSafe API, with a local Laya router choosing Laya or Jev.

    ServicebetaGo