Skip to content

Calibration and threshold setting with decision models

Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.

236 public projects · page 6 of 8

  • Python library for one-line judgment calls (likely, classify, rate) returning YES, NO or UNSURE, built for Jev and other small models.

    ModelbetaPython
  • Apache-2.0 local decision model (0.8B Qwen fine-tune on llama.cpp) behind the Jev request shape, calibrated and CPU-only.

    ServicebetaPython
  • Single-binary local cache that memoizes Jev-class decisions on model, schema and redacted state, for cheap repeats and CI replay.

    ModelbetaRust
  • Toolkit that turns Laya into a task specialist: LLM-generated and verified training data, RLCD training, calibration and longer context.

    ModelbetaPython
  • CLI and Claude Code skill that turns one typed decision into a fine-tuned, calibrated, evaluated Laya checkpoint you can serve.

    ModelbetaPython
  • Open recipe: describe a decision task, have an LLM write training data, and train a Qwen LoRA plus decision head with calibrated outputs.

    ModelbetaPython
  • Open 596M and 4B decision models that answer Choice, Score and Noul in one pass locally, with calibrated confidence for escalation.

    ServicebetaPython
  • Apache-2.0 118M multilingual decision model for Choice, Score and Noul on the System One API, with conformal abstain.

    ModelbetaPython
  • AnyJev★ 1,255

    Nokia's training-free library that turns open LLMs into Jev-style deciders robust to option order, plus self-distilled Tacit models.

    ModelresearchPython
  • nirnay★ 9

    NIRNAY 450M: open Apache-2.0 decision model fine-tuned from Laya that reports 87.9% on Banking77 with fitted ECE 0.045.

    ModelresearchPython
  • Experiment showing a RLCD-trained LoRA decision head on Qwen3.5-4B generalises only where training data covers the task structure.

    ModelresearchPython
  • Calibration harness that reads yes/no, pick-one and rating answers from one forward pass of a frozen open vision-language model.

    ModelresearchPython
  • InternLM's open multimodal decision models (0.8B-4B) trained on Qwen3.5, with calibration tooling and a 96-case calibration benchmark.

    ModelresearchPython
  • Open Apache-2.0 System One decision models (4B, 9B, 27B on Qwen) with trainer, data builders, evals and every run record.

    ServiceresearchPython
  • Preprint survey of Jev-style typed decision models: their research roots, adaptation costs and what current evidence supports.

    Modelresearch
  • Open infrastructure for training, evaluating and deploying System 1 decision models, with calibrated LoRA checkpoints on a 27B backbone.

    ModelresearchPython
  • Open stack to synthesise decision data, train and calibrate a Qwen3.5-0.8B scorer, evaluate on Mind2Web and serve Jev-style.

    ServiceresearchPython
  • Independent measurements of Jev versus open Laya on typed decisions, plus a local-first cascade that matches Jev at 1.8x speed.

    ServiceresearchPython
  • Jev-style replica: frozen Qwen3-1.7B plus a ~0.8M-parameter trained head returns calibrated Choice, Boolean and Score distributions.

    ServiceresearchPython
  • Two frozen open models answer yes/no per exam option; their agreement drives an auto-release gate and a conformal candidate set.

    ServiceresearchPython
  • Verdict: 151M ModernBERT/GLiClass non-autoregressive decision models with calibration heads, reporting wins over Jev and Laya.

    ModelresearchPython
  • Research toolkit that encodes an image once and answers several typed questions from a shared distribution, plus OpenJev-0.6B text models.

    ModelresearchPython
  • Phocinae Largha: 144.3M bilingual English/Chinese decision model on mmBERT-small with calibrated Choice, Score and Noul in one pass.

    ModelresearchPython
  • Reproducible MLX and Colab recipes that turn Qwen decoders into Jev-like typed decision models, from 0.5B on a laptop to a noncausal 27B.

    ModelresearchPython
  • Open System One model and MLX runtime returning calibrated choice, score and noul probabilities via a Jev-compatible API.

    ServiceresearchPython
  • Harness where Jev or Laya pick DOOM tactics and targets from legal options while a deterministic controller presses buttons.

    ModelresearchPython
  • Open 1.9B typed-decision model that returns an option index and calibrated probability in one forward pass, with a paper and API.

    ServiceresearchPython
  • JevBench★ 268

    MIT-licensed benchmark and leaderboard for Jev-class decision models: accuracy, calibration, latency and cost on sealed tiers.

    BenchmarkproductionPython
  • Open benchmark for System One decision models on Choice, Score and Noul, reporting accuracy, calibration and robustness separately.

    BenchmarkproductionPython
  • Agent skill that tells you when you need an eval and how to build it, with an opt-in lane routing closed-label residue to Jev.

    BenchmarkbetaPython