local-jev
Offline System One server that answers Jev-format questions with small open models, wire-compatible with TypeSafe's official SDK.
Overview
local-jev is a local, offline System One server. Software that needs a decision rather than prose, such as which queue, how severe or whether something is spam, sends text and typed questions and gets back a choice, score or yes/no with a probability for every possible answer. It speaks exactly the wire format of TypeSafe's hosted Jev API, checked with the unmodified official typesafe-sdk, so existing client code works by setting the base URL. Models run on your machine: a Qwen instruction-tuned model by default and an entailment (NLI) model as the lightweight option, and nothing leaves the machine after the first download.
On JevBench's 231 public items, the README reports the default model at 80.5% correct, against 86.6% published for hosted Jev and 81.0% for SemIf, with the gap on the hard tier. It runs on Apple Silicon, Linux or WSL2, with a GPU recommended; the default needs about 8.5 GB of memory and the smallest under 1 GB on disk. An optional browser portal is included. It is described as a research project, MIT-licensed.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which queue should this item go to?(inferred) | — |
| Score | How severe is this?(inferred) | — |
| Yes / no | Is this spam?(inferred) | — |
How it works
- Install and run local-jev serve
- Model weights download once
- Point the TypeSafe SDK at the local base URL
- Send typed questions; receive probabilities
- Optionally use the browser portal
Strengths
- Fully offline after download
- Verified against the official SDK
- Lightweight NLI option
- Published benchmark with comparison
Caveats
- Trails hosted Jev on hard items
- Default model needs about 8.5 GB memory
- Research project
Getting started
git clone https://github.com/amithgc/local-jev && cd local-jev, create a venv, uv pip install -e ., then .venv/bin/local-jev serve.At a glance
- Type
- Service
- Status
- research
- Language
- Python
- Runtime
- local HTTP server with optional web portal
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
A local, offline System One server compatible with TypeSafe.
wh000wh000/awesome-jev-liveA local, offline System One server compatible with TypeSafe's Jev API. It answers typed yes/no, category and score questions with small open models.
hellogumbo/awesome-jev
Frequently asked questions
Will the official SDK work?
Yes, it was checked with the unmodified typesafe-sdk by setting the base URL.
How accurate is it?
80.5% on JevBench's public items versus 86.6% for hosted Jev, per the README.
Does any data leave my machine?
No, after the first model download.
Related projects
Mangaba AI's local, calibrated typed-decision model for Portuguese text, trained to optimize the Brier score; proprietary since v5.
ProjectproductionPython- blink★ 22
Open one-pass typed-decision model with a dependency-free C99 runtime that also builds to a 66 KB WebAssembly module.
ServicebetaC - Foq★ 11
Local open alternative to Jev using a 1.58-bit ternary 8B model to answer booleans, choices, scores and Pydantic extraction in ~25 ms.
ProjectbetaPython Runs the open Laya decision model entirely in a browser tab with WebGPU and ONNX Runtime Web: no server, no API key.
ServicebetaTypeScriptllama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++Three open 4B decision models for English, Turkish and German that serve the Jev /v1/systemone API on a CPU in about 3 GB of RAM.
ServicebetaPython