Skip to content

Calibration and threshold setting with decision models

Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.

236 public projects · page 4 of 8

  • Resume-to-JD screening study (Chinese) with 5 Noul gates, 4 Score dimensions and a routing Choice per request, hardened over three versions.

    Projectresearch
  • Local reproduction of Jev-style inference that reads option-token logits from a small LLM in one pass, with an honest failure log.

    ProjectresearchPython
  • Small Jev measurement bench with an append-only call log, labelled claims and a stated negative result.

    ProjectresearchPython
  • Harness that runs Jev on all 19,646 TMMLU+ Traditional Chinese questions as four-way Choices, reporting accuracy and calibration.

    ProjectresearchPython
  • Tool that recomputes Jev's confidence identities on outputs published in public repositories to spot answers Jev never actually produced.

    ServiceresearchPython
  • RPG experiment pitting Jev against OpenAI's Decisions API as NPCs, comparing wins, per-move accuracy, calibration and cost.

    ServiceresearchPython
  • Korean study notes and TypeScript mock-ups of Jev concepts, with a local Laya versus GPT-5 comparison and a calibration toolkit.

    ProjectresearchTypeScript
  • Benchmark harness pitting Jev against Claude Sonnet 5 and Haiku 4.5 on Banking77 intent sorting with a confidence gate.

    ProjectresearchPython
  • Offline Python toolkit that measures sensitivity, Fisher information and information loss in the probabilities of Jev-style agent systems.

    ProjectresearchPython
  • Research code showing that short, natural-looking context additions can flip decision models like Jev to a chosen wrong answer.

    ProjectresearchPython
  • Research fork of Laya that reads typed choice and yes/no decisions from an image plus optional text in one forward pass.

    ProjectresearchPython
  • Open decision models that derive calibrated confidence from recurrent looped-layer dynamics instead of RL, with early exit and a Jev API.

    ProjectresearchPython
  • Research adding plain-English constraints to JEPA world-model planning, with Jev judging each imagined step instead of an LLM.

    ProjectresearchPython
  • Research implementation of Jev-style typed decisions over clinical evidence: supported, contradicted or unresolved.

    ProjectresearchPython
  • Local Jev-style decision model on Qwen3.5-4B with four stress tests that show where one-pass decisions break.

    ProjectresearchPython
  • LogitTrail (formerly Mini Jev): local workbench that reads typed Choice, Noul and Score decisions from a frozen LLM's logits.

    Projectresearch
  • Reproduction and stress test of a paper asking whether Jev-style probabilities stay calibrated after routing on selected examples.

    ProjectresearchPython
  • Open System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.

    ProjectresearchPython
  • openvons (open-Jev): finite-choice decision layer for text, images and Japanese voice, with a none option and execute/confirm/reject.

    ProjectresearchPython
  • Pandu: tiny local policy networks (2.9K-5K parameters) with calibrated uncertainty that fall back to a teacher when unsure.

    ProjectresearchPython
  • Laptop-scale reimplementation of RL for Calibrated Decisions: parallel constrained decoding trained with GRPO and a Brier reward.

    ProjectresearchPython
  • Stdlib-Python guardrail and model router for LLM gateways on Jev, with an independent accuracy, calibration and latency evaluation.

    ServiceresearchPython
  • Static charts comparing Jev with 110 other models on Thai standardized exams for accuracy, cost, speed and calibration.

    ProjectresearchJavaScript
  • Lightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.

    ProjectresearchPython
  • Write-up with measurements on calibrating Jev as a code reviewer: context effects, textual tells, thresholds and noise near thresholds.

    ProjectresearchPython
  • Local System One-compatible server that reads option probabilities from a small MLX LLM, with measured calibration.

    ProjectresearchPython
  • Self-hostable calibrated decision model for images and text (Qwen3-VL-4B + LoRA) scoring 82.7 percent on public JevBench.

    ProjectresearchPython
  • sapho★ 1

    Rust library and CLI for compound decisions: ask Jev small questions, combine answers with rules in a YAML/JSON decision graph.

    CLIbetaRust
  • Composable Python toolkit for Jev: question builders, confidence gates, batch evaluator, calibration, CLI, MCP server and agent skill.

    MCP serverbetaPython
  • Effect library turning Jev Choice, Noul and Score answers into typed branches with a compiler-enforced Uncertain case.

    LibrarybetaTypeScript