Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?
Paper testing, without labels, whether Jev's probabilities obey probability axioms such as P(X) + P(not X) = 1.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
An arXiv paper asking whether a typed decision model's probabilities obey the basic probability axioms, beyond being calibrated. It uses label-free coherence checks, such as comparing the probability for a statement and its negation.
Per the listing, Jev's probabilities for X and not X miss summing to one by 0.064 on average, against 0.293 for Qwen3.8-27B's first-token readout, and its single-label probabilities over-sum (to about 1.14 in one reported case).
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Paired statements X and not X, checked for coherence | yes · no |
How it works
- Construct paired and partitioned questions
- Query Jev and an LLM baseline
- Measure deviations from probability axioms
Strengths
- Checks coherence without labels
- Compares against an open LLM baseline
Caveats
- Preprint; only the listing summary was available
At a glance
- Type
- Paper
- Status
- research
Use cases and integrations
How curated lists describe it
Label-free coherence checks: Jev's probabilities for "X" and "not X" miss summing to one by 0.064 on average against 0.293 for Qwen3.8-27B's first-token readout, and its single-label probabilities over-sum to 1.14
AnotiaWang/awesome-decision-models
Frequently asked questions
What is tested?
Whether Jev's probabilities are coherent, for example P(X) + P(not X) = 1.
What was found?
An average gap of 0.064 for Jev versus 0.293 for a Qwen baseline, per the listing.
Where is it?
https://arxiv.org/abs/2609.33209
Related projects
Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPythonBlack-box reverse-engineering archive studying how the Jev decision model behaves, with evidence matrices and calibration analysis.
ProjectresearchPythonIndependent API-only audit of what Jev's probabilities mean: abstain removal, Korean, coherence, option order, interference and caching.
ProjectresearchPythonResearch framework (DART) testing whether Jev's probabilities are a reliable signal to answer, escalate or abstain.
ProjectresearchPythonEvidence ledger auditing Jev claims, recomputing published calibration and running an 800-item difficulty-gradient study.
AppresearchPythonRunnable yardstick that tests Jev's three headline claims on a small local model: typed output, no hallucination, calibration.
ProjectresearchPython