Calibration and threshold setting with decision models
Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.
236 public projects · page 4 of 8
Resume-to-JD screening study (Chinese) with 5 Noul gates, 4 Score dimensions and a routing Choice per request, hardened over three versions.
ProjectresearchLocal reproduction of Jev-style inference that reads option-token logits from a small LLM in one pass, with an honest failure log.
ProjectresearchPythonSmall Jev measurement bench with an append-only call log, labelled claims and a stated negative result.
ProjectresearchPythonHarness that runs Jev on all 19,646 TMMLU+ Traditional Chinese questions as four-way Choices, reporting accuracy and calibration.
ProjectresearchPythonTool that recomputes Jev's confidence identities on outputs published in public repositories to spot answers Jev never actually produced.
ServiceresearchPythonRPG experiment pitting Jev against OpenAI's Decisions API as NPCs, comparing wins, per-move accuracy, calibration and cost.
ServiceresearchPythonKorean study notes and TypeScript mock-ups of Jev concepts, with a local Laya versus GPT-5 comparison and a calibration toolkit.
ProjectresearchTypeScriptBenchmark harness pitting Jev against Claude Sonnet 5 and Haiku 4.5 on Banking77 intent sorting with a confidence gate.
ProjectresearchPythonOffline Python toolkit that measures sensitivity, Fisher information and information loss in the probabilities of Jev-style agent systems.
ProjectresearchPythonResearch code showing that short, natural-looking context additions can flip decision models like Jev to a chosen wrong answer.
ProjectresearchPythonResearch fork of Laya that reads typed choice and yes/no decisions from an image plus optional text in one forward pass.
ProjectresearchPythonOpen decision models that derive calibrated confidence from recurrent looped-layer dynamics instead of RL, with early exit and a Jev API.
ProjectresearchPythonResearch adding plain-English constraints to JEPA world-model planning, with Jev judging each imagined step instead of an LLM.
ProjectresearchPythonResearch implementation of Jev-style typed decisions over clinical evidence: supported, contradicted or unresolved.
ProjectresearchPythonLocal Jev-style decision model on Qwen3.5-4B with four stress tests that show where one-pass decisions break.
ProjectresearchPythonLogitTrail (formerly Mini Jev): local workbench that reads typed Choice, Noul and Score decisions from a frozen LLM's logits.
ProjectresearchReproduction and stress test of a paper asking whether Jev-style probabilities stay calibrated after routing on selected examples.
ProjectresearchPythonOpen System-1 (OpenJev): Apache-2.0 research models giving calibrated per-option probabilities, best is a 4.9B exp9a.
ProjectresearchPythonopenvons (open-Jev): finite-choice decision layer for text, images and Japanese voice, with a none option and execute/confirm/reject.
ProjectresearchPythonPandu: tiny local policy networks (2.9K-5K parameters) with calibrated uncertainty that fall back to a teacher when unsure.
ProjectresearchPythonLaptop-scale reimplementation of RL for Calibrated Decisions: parallel constrained decoding trained with GRPO and a Brier reward.
ProjectresearchPythonStdlib-Python guardrail and model router for LLM gateways on Jev, with an independent accuracy, calibration and latency evaluation.
ServiceresearchPythonStatic charts comparing Jev with 110 other models on Thai standardized exams for accuracy, cost, speed and calibration.
ProjectresearchJavaScriptLightweight reproduction of the Jev shape on a ModernBERT encoder and a diffusion LLM, comparing speed, accuracy, calibration and cost.
ProjectresearchPythonWrite-up with measurements on calibrating Jev as a code reviewer: context effects, textual tells, thresholds and noise near thresholds.
ProjectresearchPythonLocal System One-compatible server that reads option probabilities from a small MLX LLM, with measured calibration.
ProjectresearchPythonSelf-hostable calibrated decision model for images and text (Qwen3-VL-4B + LoRA) scoring 82.7 percent on public JevBench.
ProjectresearchPython- sapho★ 1
Rust library and CLI for compound decisions: ask Jev small questions, combine answers with rules in a YAML/JSON decision graph.
CLIbetaRust Composable Python toolkit for Jev: question builders, confidence gates, batch evaluator, calibration, CLI, MCP server and agent skill.
MCP serverbetaPythonEffect library turning Jev Choice, Noul and Score answers into typed branches with a compiler-enforced Uncertain case.
LibrarybetaTypeScript