jev-local
Local Jev-compatible /v1/systemone server answering Noul, Choice and Score from open-weight models, usable with the official SDK.
Overview
jev-local is a local evaluation server that implements Jev's POST /v1/systemone contract with typed noul, choice and score questions, probabilities and confidence, with no waitlist, key or closed weights. A one-line installer clones the repo, starts the API with Docker Compose and runs a smoke test, and the official SDK can point at it via base_url.
The default scorer is deterministic and carries no intelligence for contract tests; setting the scorer to hf uses a frozen open model's log probabilities, with Qwen3.5-9B as the full option (about 18 GB) and Qwen2.5-3B-Instruct as a light option for a 16 GB Mac. An optional chat-template mode helps on easier sets. The README includes a small-model leaderboard with Wilson intervals (Qwen2.5-3B-Instruct in chat mode at about 0.95 overall on one set) and a list summary mentions temperature-fit calibration with 0.83 overall on a harder set of 1,316 items.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Does this convey urgency? (example) | — |
| Other | Any Choice or Score question in the Jev request format | — |
How it works
- Install with the script or pip
- Start the server with the hf scorer and a chosen model
- Send Jev-format requests or point the SDK at it
- Read probabilities and confidence
Strengths
- No API key; runs locally
- Drop-in for the official SDK
- Contract tests and leaderboard
Caveats
- Default scorer has no intelligence
- Accuracy varies by model and task
Getting started
`curl -fsSL https://raw.githubusercontent.com/us/jev-local/main/install.sh | bash` (add `-s -- --light` for the small model).At a glance
- Type
- Library
- Status
- beta
- Language
- Python
- Runtime
- FastAPI server (Docker) with Hugging Face models
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Local reproduction: Jev-compatible `POST /v1/systemone` server answering typed `Choice`/`Score`/`Noul` questions with confidence from open weights, verified as an official-SDK drop-in with temperature-fit calibration (set3 n=1316, 0.83 overall).
yibie/awesome-jevLocal Jev-compatible evaluation server: POST /v1/systemone with typed…
wh000wh000/awesome-jev-live
Frequently asked questions
Is it Jev?
No, it is an open, local server with Jev's API shape.
Can I use the official SDK?
Yes, set base_url to the local server.
What hardware do I need?
A GPU or 24 GB+ machine for the full model, or 16 GB for the light one.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonReads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
ProjectresearchPython