jev-local
Python library that imitates Jev-style typed decisions (choice, score, yes/no) on local LLMs via Ollama, vLLM or llama.cpp.
Overview
JEV-Local is a small Python library and CLI that copies the Jev pattern for people who want to run decisions on their own hardware. You pass a state string and a set of typed questions, and the library prompts a local OpenAI-compatible model to return only JSON with a chosen option, a probability per option and a confidence value, without explanations.
It targets developers building agents or scripts who want a cheap, offline decision step instead of a cloud call. Besides the library and CLI, a setup command installs a systemd user service that exposes the decision engine as an OpenAI-compatible REST endpoint so editors and agents can call it as a model. The decision model here is whatever local LLM you point it at; the library handles prompting, parallel questions and tolerant JSON parsing.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Next step? | fill_email · fill_password · click_submit |
| Score | Score the state on a 0-10 scale | — |
| Yes / no | Yes/no gate on the given state | yes · no |
How it works
- Start a local backend such as llama-server, Ollama or vLLM
- Send a state string plus a JSON map of typed questions via CLI, library or REST
- The local model returns JSON with choice, probabilities and confidence
- Your code acts on the choice or gates on the confidence value
Strengths
- Runs fully local, so state never leaves the machine
- Several questions can be asked in one model call
- Exposes an OpenAI-compatible endpoint that editors and agents can use directly
- Minimal dependencies (httpx and pydantic)
Caveats
- Probabilities are produced by a general local LLM in its JSON output, so calibration is not guaranteed the way it is for a trained decision model
- Parts of the README are written in Turkish
- No licence stated
Getting started
pip install httpx pydantic, start a local server (for example llama-server -m qwen2.5-coder-7b-instruct-q4_k_m.gguf --port 8080), then run python3 jev_local.py --state ... --questions ... --endpoint http://localhost:8080.At a glance
- Type
- Project
- Status
- prototype
- Language
- Python
- Runtime
- CLI, library and local REST service
- Has code
- yes
Use cases and integrations
How curated lists describe it
JEV-Local: System-1 decision engine for local LLMs. Mimics TypefAI JEV: structured choices only, no chatter, calibrated confidence. Works with Ollama, vLLM, llama.cpp, LM Studio.
logicrw/awesome-jev-projects
Frequently asked questions
Does JEV-Local need a TypeSafe API key?
No. It talks only to a local OpenAI-compatible backend such as Ollama, vLLM or llama.cpp.
How is it different from real Jev?
It prompts a general local LLM to emit Jev-shaped JSON, while Jev itself is a model trained to output calibrated probabilities directly.
Which question types are supported?
Choice (pick one of named options), score on a 0-10 scale, and noul yes/no gates, several per call.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonTurns any local LLM into a Jev-style engine via masked-logit softmax, or proxies OpenAI-compatible APIs or Jev; includes a chat crush radar.
ServiceprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPython