Verdict
Apache-2.0 118M multilingual decision model for Choice, Score and Noul on the System One API, with conformal abstain.
Overview
Verdict is a small open decision model: a 118M-parameter multilingual bi-encoder covering 100+ languages that answers choose, score and check questions on CPU or in the browser via ONNX, using the same POST /v1/systemone wire format. The answer is always one of the supplied options.
It is built around three gaps the author sees in Jev and LAYA. It can be trained on your own labels in about a second on a laptop CPU; it is calibrated with temperature scaling, with reported ECE of 0.014 to 0.030; and it offers a split conformal abstain set with a coverage guarantee as its "I don't know". The README compares figures on Banking77 (0.594 zero-shot, 0.858 with 16 labels per class, 0.913 fully trained) and, per the list, honestly reports that it loses to LAYA on typed decisions (0.71 vs 0.77). It is on PyPI as verdictml with a Hugging Face demo.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Billed twice, refund or we cancel: billing, technical or sales? | billing · technical · sales |
| Score | Rate this review on a 1 to 5 scale | 1 · 2 · 3 · 4 · 5 |
| Yes / no | Does the user ask for a human? | True · False |
How it works
- Load Verdict (verdict-small) in Python or the browser.
- Ask choose, score or check questions.
- Optionally fit on your own labels in seconds.
- Use the conformal abstain set to defer uncertain cases.
Strengths
- Runs on CPU or in the browser.
- Very fast training on your own labels.
- Low reported calibration error and a conformal coverage guarantee.
Caveats
- Weaker than LAYA and Jev on some typed-decision benchmarks, per the author.
- Comparison figures are the author's own.
Getting started
pip install verdictml; from verdict import Verdict; v = Verdict()At a glance
- Type
- Model
- Status
- beta
- Language
- Python
- Runtime
- CPU, ONNX in browser
- Has code
- yes
- Has tests
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
Open alternative: Apache-2.0 118M multilingual bi-encoder that answers `Choice`, `Score`, and `Noul` questions on the same `POST /v1/systemone` wire format, calibrated with temperature scaling plus a split conformal abstain set with a coverage guarantee (ECE 0.01 to 0.03 on the public suites), runs on CPU or in the browser via ONNX, fits on your own labels in seconds, and its README says it loses to Laya on typed decisions (0.71 vs 0.77).
yibie/awesome-jev
Frequently asked questions
How big is it?
118M parameters, multilingual.
Can I train it on my data?
Yes, in about a second on a laptop CPU with 16 labels per class.
Is it better than LAYA?
Not on typed decisions overall; the author reports 0.71 vs 0.77.
Related projects
- cbjev★ 4
Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.
ProjectresearchPython 11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.
ProjectresearchPython- kev★ 9,079
Family of small Jev-like decision models you can train and run yourself, with a server compatible with TypeSafe's SDK.
ServicebetaPython Apache-2.0 local decision model (0.8B Qwen fine-tune on llama.cpp) behind the Jev request shape, calibrated and CPU-only.
ServicebetaPythonOpen 596M and 4B decision models that answer Choice, Score and Noul in one pass locally, with calibrated confidence for escalation.
ServicebetaPythonOpen Apache-2.0 System One decision models (4B, 9B, 27B on Qwen) with trainer, data builders, evals and every run record.
ServiceresearchPython