reflex
Small open decision model on a frozen Qwen3.5-4B serving /v1/systemone in about 200 ms, benchmarked against Jev on JevBench.
Overview
reflex is an open re-creation of Jev that runs on your own GPU. You give it information (a support ticket, a document, a photo with a caption) and questions with fixed answer options; it answers all of them at once with per-option percentages and never writes free text. It exposes one endpoint, POST /v1/systemone, with the same request and response shapes as TypeSafe's hosted API, answered by a single forward pass over a frozen Qwen3.5-4B in about 200 ms. A 27B configuration and a WebGPU browser demo also exist.
On JevBench public items the README reports 0.917 standard and 0.685 hard accuracy for the 4B (Jev 1.13: 0.986 and 0.730), and says official JevBench v1.2 runs placed it #5 of 36. It is unusually candid about what failed: every fine-tuning attempt, prompt optimisation, wording ensembles and a reasoning cascade were measured and dropped. What helped was the readout: a lettered yes/no answer, evidence/criterion framing and averaging two option orders. A calibration check over 1,200 exam questions measures ECE.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which team? | payments · account · other |
| Yes / no | Escalate? | — |
| Score | How urgent (0-2)? | low · medium · high |
| Yes / no | Refund requested? | — |
How it works
- Start reflex-serve on a local GPU
- POST state and typed questions to /v1/systemone
- Get per-option probabilities from one forward pass
- Apply thresholds in code, e.g. auto-route above 90% or ask a human
Strengths
- Drop-in API compatibility with Jev client code
- About 200 ms per request on a 16 GB GPU per the README
- Calibration measured with a provided check
- Documents which techniques did not help
Caveats
- Self-run benchmark numbers on a suite consulted during development
- Browser and Python configurations differ
- Needs a GPU and an 8 GB model download
Getting started
uv sync, then git checkout stable && uv run reflex-serve --stable --port 8008 and point Jev client code at http://localhost:8008.At a glance
- Type
- Service
- Status
- research
- Language
- Python
- Runtime
- PyTorch HTTP server; WebGPU demo
- Has code
- yes
- Stars
- 166
Use cases and integrations
How curated lists describe it
Small decision model on a frozen Qwen3.5-4B: one `/v1/systemone` endpoint answered by a single forward pass in about 200 ms, compared against Jev on JevBench's public items
AnotiaWang/awesome-decision-modelsA small open decision model: state + typed questions -> calibrated probabilities. A Jev / System One re-creation on Qwen3.5.
wh000wh000/awesome-jev-liveOpen-model decision engine with shared-state inference, isolated question branches, and a WebGPU demo; browser and Python configurations differ.
cobanov/awesome-jevA small open decision model: state + typed questions -> calibrated probabilities. A Jev / System One re-creation on Qwen3.5.
hellogumbo/awesome-jevA small open decision model for your own GPU: fixed answer options in, per-option percentages out, with no free text so it cannot answer off the list.
MrJev/awesome-jev
Frequently asked questions
Is reflex compatible with Jev clients?
Yes, the request and response shapes match TypeSafe's hosted API, so clients can point at localhost.
How does it compare to Jev?
The README reports lower standard and hard accuracy than Jev 1.13 for the 4B model, with the 27B closer on the hard tier.
Did fine-tuning help?
No; the README says every fine-tuning variant tried lost general judgment on long, ambiguous inputs.
Related projects
Toy local Jev-shaped decision API on a 0.5B model that answers yes/no, choice and score questions by scoring option tokens.
ServiceprototypePythonSmall open decision models (0.8B and 4B, from Qwen3.5) with a server that speaks Jev's /v1/systemone format in one forward pass.
ServicebetaPythonServes MiniCPM5-2B-8bit on Apple Silicon as a Jev-compatible /v1/systemone service and reproduces its JevBench score.
ModelprototypePythonOpen Apache-2.0 Jev-like model family on Qwen3.8-27B that reads one-token candidate preferences and serves a TypeSafe-compatible API.
ServiceresearchPythonOpen MiniCPM5-2B decision model serving /v1/systemone: typed choice, noul and score probabilities in one forward pass.
ModelresearchPythonDocker recipes and ARM64 images for serving ten native decision checkpoints, such as Cloudflare Clef, behind /v1/systemone on DGX Spark.
ServicebetaPython