Calibration and threshold setting with decision models
Measuring whether a model's probabilities can be trusted and picking per-question confidence thresholds, abstention rules and fallbacks.
236 public projects · page 1 of 8
- SkillRanker★ 134
Rust CLI that ranks which agent skills fit the next step from live session context with Jev, can abstain, and hooks into Claude Code.
CLIbetaRust Janus (formerly calibre): measures on your data the confidence threshold at which a small model like Jev should defer to a larger one.
ProjectbetaPythonClaude Code PreToolUse layer where Jev answers YAML-defined questions about each tool call, with 19 hard rules and calibration runs.
ProjectbetaTypeScriptInstrumented 2048 web lab where every move is a Jev Choice over four directions, with probabilities, latency and cost shown live.
ProjectbetaTypeScriptOpenAI-compatible endpoint that answers an agent's command-approval call with APPROVE, DENY or ESCALATE from a Jev Choice, failing closed.
ProxybetaPythonAgent skills for placing, building, evaluating and improving decision-model systems, with Jev as the default hosted example.
Agent skillbetaClaude Code plugin that sends judgment calls to Jev, acts at 0.8+ for yes or 0.2- for no, and gathers more evidence in between.
Agent skillbetaJavaScriptClaude Code plugin where Jev decides and Claude writes: sort piles of text, calibrate thresholds, route prompts and gate Bash commands.
PluginbetaClaude Code plugin and CLI for the Jev funnel: judge items in bulk, settle confident ones in code, send only uncertain ones onward.
PluginbetaPythonUnofficial Claude Code plugin that checks 'done' claims and small judgment calls with Jev, logs receipts, and can warn on unverified stops.
PluginbetaRust CLI and library that asks Jev typed questions via OpenRouter and combines the probabilities with fuzzy-logic rules into decisions.
CLIbetaRustGitHub Action that triages issues with Jev, applying label, spam, needs-info and duplicate tags only above set thresholds.
ProjectbetaTypeScriptUnofficial standard-library Go client for Jev with generic typed Choice values and local confidence thresholds.
ProjectbetaGoRust CLI for Jev: ask yes/no, choice and rubric questions about text and branch on exit codes, JSON or MCP tools.
MCP serverbetaRustNautilus Compass agent memory layer whose jev-trust middleware logs Jev decisions and measures calibration in your domain.
ServicebetaPythonPortuguese-language agent skill for millisecond Jev decisions via OpenRouter, with validation, an MCP mode and calibration evals.
Agent skillbetaJavaScriptEditorial QA tool where Jev judges per fact whether an edit preserved the original, with a confidence gate and human review fallback.
ProjectbetaJavaTypeScript library adding policy, confidence gates, shadow mode, recipes and an eval CLI around Jev answers.
CLIbetaTypeScriptNatural-language linter combining ast-grep selectors with Jev to catch misleading names, stale comments and tests that miss their claims.
ProjectbetaTypeScriptEval-first MCP server for Jev: prototype questions, map them over many items, and measure accuracy and thresholds on labelled data.
MCP serverbetaTypeScriptRust MCP server for Codex that hands classification, relevance, candidate selection and evaluation to Jev with accept/verify policies.
MCP serverbetaRustOptimises the instructions and criterion descriptions of a Jev Choice from labelled data with EvoPrompt or GEPA, scored by Brier.
ProjectbetaPythonToolkit for Jev decision contracts: compiled policies, egress controls, tracing, stability checks and certified decision SLOs.
ProjectbetaTypeScriptDecision-forensics library that explains a Jev answer by ablating parts of the input and measuring how far the probability moves.
ProjectbetaPythonpytest-style tool that pins fixtures and expected Jev answers, then fails a model upgrade when answers flip or confidence drops.
ProjectbetaPythonCLI that has Jev judge a video's subtitles segment by segment, then cuts a highlight reel and a filler-free full version.
ProjectbetaPythonTest harness for any Jev-API decision model: accuracy, calibration and robustness probes for reordering, injection and padding.
ServicebetaPythonPython library that replaces hard-coded RAG thresholds with calibrated decisions for stopping, chunking, selection, abstention and caching.
ServicebetaPythonSQLite and Node library adding Jev judgments to SQL for matching, evidence picking, ranking, review queues and semantic CI checks.
ProjectbetaJavaScriptRecord linkage for researchers: write the match rule in plain English and Jev judges each candidate pair with a probability.
ProjectbetaPython