Skip to content

Calibration and threshold setting with decision models

Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.

236 public projects · page 2 of 8

  • One YAML task to classify, route, score or verify text with rules, Jev, NanoJev or an OpenAI-compatible model, with cost records.

    CLIbetaPython
  • Records the full distribution of every Jev answer by state, question and model, so thresholds can move over history with no new calls.

    LibrarybetaPython
  • Unix-pipeable CLI for Jev and other System One models: ask questions, gate shell commands via exit codes, calibrate thresholds.

    CLIbetaGo
  • Auto mode for Pi: deterministic policy first, then Jev judges escalated bash/write/edit calls, failing closed when undecidable.

    ProjectbetaTypeScript
  • TypeScript library for typed semantic decisions via Zod schemas or native batches, defaulting to Jev, with confidence-gated policies.

    LibrarybetaTypeScript
  • Bilingual turtle-soup (lateral-thinking) puzzle site where Jev acts as host, judging player questions and grading solutions.

    AppbetaTypeScript
  • MCP server and agent skill for designing Jev judgments: question linting, batch evaluation, threshold calibration and reranking.

    MCP serverbetaTypeScript
  • Monitoring proxy and client that records Jev answer probabilities and alerts on version changes, drift, edge crowding or accuracy drops.

    ProxybetaPython
  • Multiplayer GitHub triage dashboard where Jev answers kind, severity, urgency, duplicate and next step per issue.

    Appbeta
  • Bun-based MCP server wrapping Jev with compact answers and a code-computed act, review or abstain verdict.

    MCP serverbetaTypeScript
  • web4kit starter site whose home page is planned per visitor by a System One model (Jev or local LAYA), with calibrated fallback to rules.

    ProjectbetaTypeScript
  • Offline linter and agent skill that checks Jev requests for badly designed questions before they come back confidently wrong.

    Agent skillbetaTypeScript
  • Experimental lab for semantic business gates: each gate is an independent Jev Noul that code maps to NO, REVIEW or YES by threshold.

    Projectprototype
  • Adapter that turns existing LLMs into Jev-style decision functions with option scores, optional calibration and review thresholds.

    ProjectprototypePython
  • Decision sidecar and Copilot CLI pre-turn hook that mimics Jev choice, noul and score using Azure Foundry Cohere Rerank.

    HookprototypePython
  • Decision engine that combines Jev probability distributions with loss matrices to decide when a clarifying question is worth asking.

    ProjectprototypeTypeScript
  • Natural-language SQLite interface where code proposes options, Jev chooses among them, and a compiler builds deterministic SQL.

    ProjectprototypePython
  • CLI, MCP server and skill for issue triage where Jev predicts issue kind, affected module and whether information is sufficient.

    MCP serverprototypePython
  • Decision-first CSV cleaning: code proposes repairs, Jev scores action, applicability and risk, a fixed policy applies or abstains.

    ProjectprototypePython
  • Neo4j demos using Jev as a calibrated judge in GraphRAG pipelines, starting with entity resolution and a review queue.

    AppprototypePython
  • Browser Minesweeper demo where Jev answers 'is this cell a mine?' for every frontier cell in one request, with an exact solver as fallback.

    ProjectprototypeJavaScript
  • Python library to batch, gate, trace and calibrate thousands of typed Jev decisions, maximising questions per call.

    ProjectprototypePython
  • Agent skill and CLI that chains Jev question design: find decision calls, check fit, draft, lint, calibrate thresholds and plan a cascade.

    Agent skillprototypePython
  • Autonomous PR review bot with typed Jev questions, deterministic approval gates and escalation of uncertain reviews to trusted owners.

    ProjectprototypePython
  • Stdlib Python dispatcher turning typed function signatures into one batched Jev request, with a browser harness against ground truth.

    ProjectprototypePython
  • Paper market-making system where code builds the state and policy while Jev answers six atomic judgments per 300 ms block.

    ProjectprototypePython
  • Tool that lets code and agents ask Jev typed questions via reusable jevels, mapping answers to act, mark or fall_back.

    ProjectprototypeTypeScript
  • Python sandbox turning Jev issue-category and reproduction judgments into testable routing proposals, with offline replay.

    ProjectprototypePython
  • Playbook for using Jev as a cheap gate before expensive agent steps: confidence policy, question recipes, a skill and a router.

    Projectprototype
  • Two-tier macOS decision engine: a MiniLM encoder on the Neural Engine decides, escalating unsure cases to Apple's on-device model.

    ProjectprototypePython