Open Medical Jev
Two frozen open models answer yes/no per exam option; their agreement drives an auto-release gate and a conformal candidate set.
Overview
Open Medical Jev aims for Jev-class judgment on medical licensing exam questions through computation rather than training. Two untouched open models, Qwen3.5-27B and Qwen3.5-35B-A3B via GGUF, each answer a yes/no judgment per answer option. A small fit-free router turns their agreement into a combined confidence (discounted when they disagree), applies Chow's rule as an auto-release gate so confident items are answered and the rest go to human review, and builds a split-conformal candidate set with a guaranteed error rate.
The README reports results within about 2 points of hosted Jev on three 600-item national licensing exams, with no fine-tuning, distillation or corpus. Three compute modes are offered: fast at about 0.076 s per item, and general and high cascades that release 84% and 66% of items at 93% and 97% precision while cutting compute by about 63% and 49%. Because nothing is trained, swapping in a newer model only needs refitting two constants. Figures are author-reported.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this answer option correct for the exam question? | yes · no |
| Yes / no | Is combined confidence high enough to auto-release? | release · review |
How it works
- Each frozen reader answers yes/no per option.
- Combine agreement into a confidence score.
- Auto-release above the gate; escalate the rest to human review.
- Return a conformal candidate set at the chosen error rate.
Strengths
- No training; model-agnostic.
- Statistical guarantee via conformal prediction.
- Cascades reduce compute substantially.
Caveats
- Author-reported results.
- Requires running two large quantised models.
- Medical use needs human oversight.
At a glance
- Type
- Service
- Status
- research
- Language
- Python
- Runtime
- llama.cpp (GGUF), service
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Medical evaluation: two frozen local readers answer one Noul-style yes/no probability per exam option, a fit-free router auto-releases items above the combined-confidence gate and escalates the rest, and a split-conformal candidate set bounds the error - landing within 2 points of hosted Jev on three 600-item national licensing exams with no fine-tuning, no distillation and no corpus.
yibie/awesome-jevTwo frozen off-the-shelf models answer a yes/no judgment per option, and their agreement becomes a confidence, an auto-release gate and a guaranteed candidate set. Nothing is trained. Its figures are measured against Jev 1.13.0 on national medical exams and are author-reported. Chinese and English.
MrJev/awesome-jev
Frequently asked questions
Is anything trained?
No, it uses two frozen open models plus a small router.
How does it compare to Jev?
Within about 2 points on three national medical exams, per the README.
What happens to low-confidence items?
They are escalated to human review rather than auto-released.
Related projects
- cbjev★ 4
Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.
ProjectresearchPython Joke decision model trained to answer wrongly with total confidence: 1.9% accuracy at 0.957 mean confidence, via a Jev-style API.
ProjectresearchPythonOpen clinical text classifiers (0.8B-9B on Qwen3.5) with labels set at inference time, run locally and compared with Jev 1.13.
ProjectresearchPythonPre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPython11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.
ProjectresearchPythonTiny pre-registered experiment testing whether a confidence-scoring reward (RLCD-like) yields confidences safe to automate on at 0.9.
ProjectresearchPython