Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

poorjev

Local Jev alternative on zero-shot NLI models with temperature scaling and conformal abstention; reports ECE 0.170 to 0.071, offline.

Servicebeta
View on GitHub ↗

Overview

poorjev, the poor man's Jev, is an open-source, local-first System One decision layer for LLM apps. It implements Jev's typed Choice, Score and Noul interface on commodity zero-shot NLI models and adds temperature scaling and conformal abstention so stated confidences are honest; the README reports that on its shipped eval set calibration error fell from 0.170 to 0.071 with no loss of accuracy, cross-validated per the list summary. It runs offline after one download, with no API key or waitlist.

One call answers several typed questions in one pass, returning values that are always among your options with calibrated confidence. An MCP server exposes gate (should this action be blocked), judge, classify, rate and decide tools to Claude Code or Claude Desktop, so an agent can gate risky tool calls locally at no token cost. It suits developers who want calibrated local decisions without a hosted service.

What the decision model decides

TypeQuestionAnswers
Choicetopicbilling · technical · account · shipping · other
Scorefrustrationlow · medium · high
Yes / noThe customer needs a response today.true · false
Yes / nogate: should this action be blocked?yes · no

How it works

  1. pip install poorjev[local].
  2. Call client.ask with state and typed questions.
  3. NLI model scores options; temperature scaling calibrates.
  4. Abstain via conformal thresholds when unsure.

Strengths

  • Runs offline with no key.
  • Calibrated confidence with reproducible eval.
  • MCP tools for Claude Code.

Caveats

  • Zero-shot NLI models are weaker than larger decision models.
  • Calibration measured on its own labelled set.

Getting started

pip install "poorjev[local]"; for Claude Code: pip install "poorjev[local,mcp]" && claude mcp add poorjev -- poorjev serve.

At a glance

Type
Service
Status
beta
Language
Python
Runtime
library, MCP server
Has code
yes
Stars
10

Use cases and integrations

How curated lists describe it

  • Local reproduction: implements Jev's typed `Choice`/`Score`/`Noul` interface on commodity zero-shot NLI models and makes the confidence honest with temperature scaling and conformal abstention, shipping a reproducible calibration eval (ECE 0.170 to 0.071, cross-validated) that runs offline with no API key.

    yibie/awesome-jev

  • Open-source, local Jev alternative: a System One decision layer with provably calibrated confidence (ECE 0.170→0.071). Typed decisions, runs offline, no API key, no waitlist.

    wh000wh000/awesome-jev-live

  • Local-first reproduction of the Choice/Score/Noul interface on commodity zero-shot NLI models, adding temperature scaling and conformal abstention; ships a reproducible calibration eval (ECE 0.170 to 0.071 on its own labeled set) and runs offline with no API key.

    cobanov/awesome-jev

  • Open-source, local Jev alternative: a System One decision layer with provably calibrated confidence (ECE 0.170→0.071). Typed decisions, runs offline, no API key, no waitlist.

    hellogumbo/awesome-jev

Frequently asked questions

Does it need an API key?

No, it runs locally after one download.

How calibrated is it?

The README reports ECE falling from 0.170 to 0.071 on its eval set.

Can Claude Code use it?

Yes, through its MCP server with gate, judge, classify, rate and decide tools.

Related projects

  • laya-self-train-prime

    CLI and Claude Code skill that turns one typed decision into a fine-tuned, calibrated, evaluated Laya checkpoint you can serve.

    ModelbetaPython
  • cbjev★ 4

    Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.

    ProjectresearchPython
  • bev

    Joke decision model trained to answer wrongly with total confidence: 1.9% accuracy at 0.957 mean confidence, via a Jev-style API.

    ProjectresearchPython
  • decision-model-audit

    Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.

    ProjectresearchPython
  • ej

    11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.

    ProjectresearchPython
  • generative-ai-jev-rlcd-calibrated-decisions

    Tiny pre-registered experiment testing whether a confidence-scoring reward (RLCD-like) yields confidences safe to automate on at 0.9.

    ProjectresearchPython