Calibration and threshold setting with decision models
Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.
236 public projects · page 6 of 8
Python library for one-line judgment calls (likely, classify, rate) returning YES, NO or UNSURE, built for Jev and other small models.
ModelbetaPythonApache-2.0 local decision model (0.8B Qwen fine-tune on llama.cpp) behind the Jev request shape, calibrated and CPU-only.
ServicebetaPythonSingle-binary local cache that memoizes Jev-class decisions on model, schema and redacted state, for cheap repeats and CI replay.
ModelbetaRustToolkit that turns Laya into a task specialist: LLM-generated and verified training data, RLCD training, calibration and longer context.
ModelbetaPythonCLI and Claude Code skill that turns one typed decision into a fine-tuned, calibrated, evaluated Laya checkpoint you can serve.
ModelbetaPythonOpen recipe: describe a decision task, have an LLM write training data, and train a Qwen LoRA plus decision head with calibrated outputs.
ModelbetaPythonOpen 596M and 4B decision models that answer Choice, Score and Noul in one pass locally, with calibrated confidence for escalation.
ServicebetaPythonApache-2.0 118M multilingual decision model for Choice, Score and Noul on the System One API, with conformal abstain.
ModelbetaPython- AnyJev★ 1,255
Nokia's training-free library that turns open LLMs into Jev-style deciders robust to option order, plus self-distilled Tacit models.
ModelresearchPython - nirnay★ 9
NIRNAY 450M: open Apache-2.0 decision model fine-tuned from Laya that reports 87.9% on Banking77 with fitted ECE 0.045.
ModelresearchPython Experiment showing a RLCD-trained LoRA decision head on Qwen3.5-4B generalises only where training data covers the task structure.
ModelresearchPythonCalibration harness that reads yes/no, pick-one and rating answers from one forward pass of a frozen open vision-language model.
ModelresearchPythonInternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.
ModelresearchPythonOpen Apache-2.0 System One decision models (4B, 9B, 27B on Qwen) with trainer, data builders, evals and every run record.
ServiceresearchPythonPreprint survey of Jev-style typed decision models: their research roots, adaptation costs and what current evidence supports.
ModelresearchOpen infrastructure for training, evaluating and deploying System 1 decision models, with calibrated LoRA checkpoints on a 27B backbone.
ModelresearchPythonOpen stack to synthesise decision data, train and calibrate a Qwen3.5-0.8B scorer, evaluate on Mind2Web and serve Jev-style.
ServiceresearchPythonIndependent measurements of Jev versus open Laya on typed decisions, plus a local-first cascade that matches Jev at 1.8x speed.
ServiceresearchPythonJev-style replica: frozen Qwen3-1.7B plus a ~0.8M-parameter trained head returns calibrated Choice, Boolean and Score distributions.
ServiceresearchPythonTwo frozen open models answer yes/no per exam option; their agreement drives an auto-release gate and a conformal candidate set.
ServiceresearchPythonVerdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.
ModelresearchPythonResearch toolkit that encodes an image once and answers several typed questions from a shared distribution, plus OpenJev-0.6B text models.
ModelresearchPythonPhocinae Largha: 144.3M bilingual English/Chinese decision model on mmBERT-small with calibrated Choice, Score and Noul in one pass.
ModelresearchPythonReproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.
ModelresearchPythonOpen System One model and MLX runtime returning calibrated choice, score and noul probabilities via a Jev-compatible API.
ServiceresearchPythonHarness where Jev or Laya pick DOOM tactics and targets from legal options while a deterministic controller presses buttons.
ModelresearchPythonOpen 1.9B typed-decision model that returns an option index and calibrated probability in one forward pass, with a paper and API.
ServiceresearchPython- JevBench★ 268
MIT-licensed benchmark and leaderboard for Jev-class decision models: accuracy, calibration, latency and cost on sealed tiers.
BenchmarkproductionPython - reflexbench★ 6
Open benchmark for System One decision models on Choice, Score and Noul, reporting accuracy, calibration and robustness separately.
BenchmarkproductionPython Agent skill that tells you when you need an eval and how to build it, with an opt-in lane routing closed-label residue to Jev.
BenchmarkbetaPython