jevmlx
Scores every allowed option of a typed schema from one MLX prefill on Apple Silicon and assembles schema-valid JSON with probabilities.
Overview
jevmlx brings Jev-style typed decisions to local models on Apple Silicon. You give it a schema of fields (booleans, enums and multi-selects) and a context string; it runs one batched prefill on a local MLX model, reads one logits vector per field, applies a restricted softmax over the allowed options, and assembles the JSON from the winners with a probability per field. No text is generated, so the output is valid by construction.
It offers a CLI, a Python library and an HTTP server with a /decide endpoint. Model aliases map to 4-bit Qwen2.5 Instruct models (7B 'quality' by default, 3B 'fast', 1.5B for tests), and any MLX Hub id works. The README gives schema design advice: options must be mutually exclusive, add an escape option such as other or NONE_OF_ABOVE when the list may not be exhaustive, and mark scales as ordered. It is aimed at Mac developers who need local classification without structured-output retries.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which enum value fits this field given the context? | — |
| Yes / no | Is this boolean field true for the context? | — |
| Score | Where on an ordered scale does this field fall? | — |
How it works
- Define a schema of boolean, enum and multi-select fields
- Send schema and context via CLI, library or POST /decide
- One prefill scores every option of every field from logits
- JSON is assembled from the winners with per-field probabilities
Strengths
- Runs fully local on Apple Silicon
- Schema-valid output without retries
- All fields scored in one forward pass
- Clear guidance on escape options and ordered scales
Caveats
- Apple Silicon Mac (M1+) and Python 3.12+ only
- Accuracy depends on the underlying general instruct model
- First use downloads 2 to 4.5 GB of weights
Getting started
git clone https://github.com/bnsd55/jevmlx && cd jevmlx && ./setup.sh, then jevmlx decide --preset support_triage --json or jevmlx serve --model fast --port 8000.At a glance
- Type
- Model
- Status
- beta
- Language
- Python
- Runtime
- MLX on Apple Silicon; CLI and HTTP server
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Jev-style parallel constrained decisions for any MLX model on Apple Silicon: schema-valid JSON in one forward pass
AnotiaWang/awesome-decision-modelsJev-style parallel constrained decisions for MLX models on Apple Silicon.
cobanov/awesome-jevJev-style parallel constrained decisions for any MLX model on Apple Silicon. Typed, schema-valid JSON in one forward pass.
hellogumbo/awesome-jevJev-style typed decisions from local MLX models on Apple Silicon.
MrJev/awesome-jev
Frequently asked questions
How is this different from structured output?
Structured output generates JSON token by token and retries on failure; jevmlx scores the finite options from logits and builds the JSON itself.
Which models does it use?
4-bit Qwen2.5 Instruct models by alias, or any MLX model id from the Hugging Face Hub.
Does it call TypeSafe?
No, it runs local models only.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonReads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
ProjectresearchPython