SystemNone
Gateway that distils System One or LLM answers into small fine-tuned classifiers and serves them when confident, falling back otherwise.
Overview
SystemNone is a gateway that sits in front of a System One model such as Jev or an LLM and gradually replaces it with small fine-tuned classifiers. A Rust gateway serves a System One-style classification API, logs every input, answers with a distilled student when the student is confident, and otherwise calls the configured System One model or LLM. A Python library and worker judge logged inputs, fine-tune students such as DeBERTa-v3, ModernBERT or Ettin, calibrate them and export to ONNX.
It targets teams with recurring classification traffic but no in-house fine-tuning expertise. Students are trained once a prompt reaches a set volume (10,000 requests by default), confidence is calibrated on held-out data and each prompt's serving threshold is chosen to meet a target accuracy. The README's banking77 comparison reports a 17M Ettin student at 91.95% with sub-millisecond CPU latency, against 79.4% for Jev. It notes TypeSafe's terms forbid training on its outputs, so Jev can serve as the fallback model but not as the judge.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which class does this input belong to (prompt-defined labels)?(inferred) | — |
| Yes / no | Is the student classifier confident enough to answer instead of the fallback model?(inferred) | — |
How it works
- Clients call the gateway's classification API.
- The gateway logs inputs and answers via the System One model or LLM.
- A judge labels logged inputs and the worker fine-tunes a student once volume is reached.
- The calibrated student serves answers above its threshold; below it, the fallback model answers.
Strengths
- Lower latency and cost once students take over.
- Same API for clients throughout.
- Per-prompt thresholds tied to a target accuracy.
Caveats
- Student accuracy is capped by the judge's accuracy.
- TypeSafe outputs cannot be used as training labels under its terms.
- No licence for the code stated in the README excerpt.
Getting started
Install uv and Rust, then run make to start the gateway, training worker and dashboard locally.At a glance
- Type
- Service
- Status
- prototype
- Language
- Rust
- Runtime
- gateway + Python training worker
- Has code
- yes
- Has tests
- yes
- Stars
- 0
Use cases and integrations
How curated lists describe it
System None is a gateway for System One models that speeds up inference by fine-tuning classifiers automatically
wh000wh000/awesome-jev-live
Frequently asked questions
What does SystemNone decide?
It classifies inputs into prompt-defined labels and decides whether a cheap student or the larger fallback model should answer.
Can Jev be the judge?
No; the README says TypeSafe's terms forbid training on its outputs, so Jev can only be the fallback System One model.
How do I run it?
Install uv and Rust and run make; a TypeSafe key is optional.
Related projects
OpenAI-compatible LLM router across 80+ providers with an optional jev-auto mode that picks the model using calibrated probabilities.
LibrarybetaTypeScriptGo model-routing gateway for OpenAI and Anthropic clients where Jev picks a model from editable model cards, with CLI and dashboard.
CLIbetaGo- Jevstiller★ 68
Proxy that distils a repeated Jev classification task into a local model on the fly and answers most requests itself once it agrees.
ModelbetaPython Inception's decision model on OpenRouter, with a free route, cited by Inception at up to 14 decisions per second.
ModelLocal-first routing engine for long-running agent tasks that uses a Jev decider to pick a model group per turn.
PluginproductionRust, TypeScriptOpen-source AI gateway for 100+ LLMs that can use Jev to classify requests for complexity routing and to compact tool results.
ProxyproductionPython