jevinf
Inference engine for Jev-style decision models using segmented forwards with prefix reuse, serving the Jev wire contract.
Overview
jevinf is an inference engine for decision models of the Jev kind. Each candidate answer path runs as segmented forward passes with prefix reuse, and the Jev wire contract is served on top, so any Jev client, including the official SDK, works by pointing TYPESAFE_BASE_URL at it. NanoJev is the backend wired up today; the checkpoint and evaluation split are not in the repository, but exact download commands and file hashes are pinned in AGENTS.md.
The README reports throughput gains from the arrangement of forwards rather than the device: on Apple MPS, 2.57 times on a single API-sized request (25.80 to 10.06 seconds) and 2.27 times end to end on the dev split (84.2 to 37.0 seconds) at 100% argmax agreement, and 2.15 and 2.07 times on CUDA. Commands include a self-check, benchmarks, evaluation on a request or split, an oracle baseline and a comparison of outputs. The server exposes /v1/systemone and /api/evaluate, optionally enforcing a bearer key. A strategies document explains the shape dependence behind the numbers. Documentation is in English and Chinese, MIT licensed.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Serves arbitrary Jev-shaped Choice, Score and Noul requests from a local decision model | — |
How it works
- Download NanoJev weights and dev split (pinned hashes)
- Run selfcheck and benchmarks
- Serve /v1/systemone locally
- Point Jev clients at the local base URL
Strengths
- About 2-2.6x throughput from forward arrangement per the README
- 100% argmax agreement with the baseline
- Works with the official SDK
Caveats
- Weights not included in the repo
- Currently one backend (NanoJev)
Getting started
uv sync, then `uv run jevinf serve -m models/NanoJev --port 8226` and set TYPESAFE_BASE_URL=http://127.0.0.1:8226.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Local inference server (MPS / CUDA)
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
An inference engine for decision models of the Jev kind: each candidate path runs as segmented forwards with prefix reuse, and the Jev wire contract is served on top. NanoJev is the backend wired up today.
logicrw/awesome-jev-projects
Frequently asked questions
Can I use the official TypeSafe SDK with it?
Yes, set TYPESAFE_BASE_URL to the local server.
How much faster is it?
About 2.57x on MPS and 2.15x on CUDA for single requests, at identical argmax answers.
Which model does it run?
NanoJev today, downloaded separately.
Related projects
Native Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoTurns any chat model on SGLang, vLLM, transformers or MLX into a Jev-compatible service with one prefill and one label token per question.
ProjectbetaPythonSelf-hosted Jev-compatible System One server that sits in front of llama-server and reads option probabilities from any GGUF model.
ServicebetaPythonResearch preview: Python client and pinned adapters for running System One decision models on your own Modal GPUs.
LibraryresearchPython- simple-jev★ 602
Featherless AI server that turns compatible open Hugging Face models into a /v1/systemone classifier by reading next-token logits.
ServicebetaPython