Calibration and threshold setting with decision models
Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.
236 public projects · page 8 of 8
Repository on a direct approach to Jev-style models using Reinforcement Learning for Calibrated Decisions (RLCD).
ProjectClaude Code and Codex skills powered by Jev, published with a calibration curve measured on 4,995 real agent decisions.
ProjectDependency-free TypeScript SDK that turns Jev answers into named decisions with hysteresis thresholds, journaling and replay.
LibraryTypeScriptEducational site with six bounded recipes and a custom builder for AI SDK Choice, Score and Boolean evaluations, exportable as TypeScript.
ServiceTypeScriptPython trust middleware for the Jev API that logs decisions, measures calibration from your recorded outcomes and signs the evidence.
ServicePythonOne shared online chess board where the internet plays against Jev, with every legal move as an option and live calibration scoring.
DemoprototypeIndependent leaderboard grading Jev and six LLMs on the same typed questions against human labels, with open per-decision data.
BenchmarkresearchPaper testing, without labels, whether Jev's probabilities obey probability axioms such as P(X) + P(not X) = 1.
PaperresearchPaper evaluating zero-shot Jev 1.13 on 37 datasets against Qwen and Gemma option probabilities, with code and raw responses released.
PaperresearchPaper testing Jev, Laya, Decider and Nimble against classifiers and LLM judges on prompt injection and risk screening.
PaperresearchPaper introducing PixelJev, a native-image decision interface on small open multimodal models with few-shot adaptation.
PaperresearchPaper auditing 28 early Jev papers and proposing a 14-item checklist for evaluating typed decision models.
PaperresearchIndependent paper testing Jev's self-reported fields against ground truth, finding a collapsed verdict vocabulary and misleading flags.
PaperresearchOfficial TypeSafe docs page explaining how Jev's confidence is derived from the probability distribution and how to set thresholds.
ArticleproductionTypeSafe's documentation of where Jev 1.13 is known to fail: counting, dates, indirection, distractors, adversarial state and more.
ArticleproductionTypeSafe docs page on decision patterns: speculative fan-out, confidence-gated routing, composite scoring and intent routing.
ArticleproductionTypeSafe cookbook on why repeated Jev calls can differ, showing how an abstention rule raises repeat agreement.
ArticleproductionIndependent early-access trial of Jev that measures its calibrated judgments and reports cost per decision.
ArticleresearchBlog post testing Jev in 16,000 calls against two OpenAI models, covering where it wins, where it breaks and how to set thresholds.
ArticleresearchArticle cross-checking 108 statements to argue that Jev's calibration is not unique, including the author's own corrections.
ArticlediscussionFlavio Copes' hands-on walkthrough of Jev and the System One idea, aimed at developers who have only seen LLM marketing.
ArticlediscussionBlog post by Alex Molas with an HN thread arguing statistically that Jev's calibration claim cannot hold, with methods written out.
ArticlediscussionIndependent multilingual guide site covering Choice, Score, Noul, Python SDK requests, thresholds, fallbacks and human escalation.
ArticlePythonReddit thread arguing Jev's type-safety guarantee is real but its calibration claim lacks a published ECE or reliability curve.
DiscussiondiscussionVals AI's independent benchmark of Jev: matched a frontier model on claim verification at a fraction of cost, but ranked last on LegalBench.
DiscussiondiscussionHacker News thread on RLCD, the training approach described as behind Jev's calibrated decisions.
Discussiondiscussion