Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

arbiter

Serving layer that runs typed-decision models like Laya on your GPU or Mac behind a Jev-compatible /v1/systemone API.

Servicebeta
View on GitHub ↗

Overview

arbiter is a small HTTP server for typed-decision models, which answer questions about a piece of text in one forward pass with no generated tokens, returning an answer and a probability distribution per question. It speaks TypeSafe's Jev API, so a Jev client works against it by changing the base URL, and adds routing, latency, confidence and action fields that Jev clients ignore.

It is deliberately only the serving layer: cross-request micro-batching, routing between checkpoints, a playground, and the measurements behind every default. The model is pluggable; today it serves Laya with three checkpoints (English, multilingual and a typed-decisions fine-tune) resident at once with automatic routing between them. Recipes exist for NVIDIA GPUs and Apple machines, plus MCP and Claude Code integrations. The author's blog post explains it was built as a local Jev.

What the decision model decides

TypeQuestionAnswers
Yes / noThis message needs a human rather than an automated reply.yes · no

How it works

  1. Point a Jev client at the local base URL
  2. Server micro-batches requests
  3. Routes each request to the right Laya checkpoint
  4. Returns Jev-shaped answers plus routing, latency and confidence

Strengths

  • Drop-in local replacement for the Jev API
  • No per-call cost or data leaving the machine
  • Measured defaults and hardware recipes

Caveats

  • Model quality is Laya's, not Jev's
  • Requires a GPU or Apple silicon

Getting started

Follow the recipe for your hardware under recipes/nvidia or recipes/apple, then call POST /v1/systemone.

At a glance

Type
Service
Status
beta
Language
Python
Runtime
local HTTP server (NVIDIA GPU or Mac)
Has code
yes

Use cases and integrations

How curated lists describe it

  • Serves typed-decision models, Laya or your own, behind a Jev-compatible API on an NVIDIA GPU or a Mac.

    MrJev/awesome-jev

Frequently asked questions

Can my existing Jev client use it?

Yes, change the base URL; extra response keys are ignored by Jev clients.

Which model does it serve?

Laya today, with three checkpoints routed automatically; other engines can be plugged in.

Does it need a TypeSafe key?

No, it runs locally.

Related projects

  • opendecider

    Open calibrated decision models from 400M to 80B with Jev-compatible server, MCP server and clients, benchmarked against Jev.

    ModelbetaPython
  • brier

    Mangaba AI's local, calibrated typed-decision model for Portuguese text, trained to optimize the Brier score; proprietary since v5.

    ProjectproductionPython
  • blink★ 22

    Open one-pass typed-decision model with a dependency-free C99 runtime that also builds to a 66 KB WebAssembly module.

    ServicebetaC
  • Foq★ 11

    Local open alternative to Jev using a 1.58-bit ternary 8B model to answer booleans, choices, scores and Pydantic extraction in ~25 ms.

    ProjectbetaPython
  • browser-laya

    Runs the open Laya decision model entirely in a browser tab with WebGPU and ONNX Runtime Web: no server, no API key.

    ServicebetaTypeScript
  • ifReflex.cpp

    llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.

    ProjectbetaC++