Best Calibrated Decision Models
A well-calibrated decision model is right about 80% of the time when it says 80%, so its probabilities can be used as thresholds directly. This ranking sorts models at or above the median Decision Index score by raw expected calibration error (lower is better), with accuracy as the tie-break, so a model cannot rank by being uniformly unsure.
Ranked list
Scores, calibration (ECE) and latency from the Jev Decision Index; cost per 1,000 decisions at 200 input tokens each. Updated daily.
More decision-model rankings
Frequently asked questions
Which decision model is best calibrated?
Calibration measurements are being loaded; this ranking fills in once they are.
What is expected calibration error (ECE)?
ECE groups predictions by stated confidence and measures the average gap between confidence and accuracy in each group. 0 is perfect; below about 0.05 is good enough to use probabilities as thresholds.
Can calibration be fixed after the fact?
Often, yes: fitting one temperature per question type on a small labelled set usually lowers ECE a lot. The ranking here uses raw ECE as published by the Decision Index.