jev-pr-judge
Next.js app and GitHub Action where one parallel Jev call judges whether a pull request does what it claims, with policy in code.
Overview
PR Judge demonstrates TypeSafe's System One primitives judging whether a GitHub pull request actually does what it claims, as a fast typed verdict for developers running coding agents instead of an LLM-as-judge prompt chain. Every question runs in parallel against one bounded state, returning scores, yes/no probabilities and a labelled choice.
Weights, thresholds and hard rules live in TypeScript policy code that is reviewable and unit-tested, so the model answers questions and code decides. The final decision is approve, human_review or send_back, with profiles for balanced, hotfix, refactor and docs changes. It runs as a local web app where you paste a PR URL, or as a GitHub Action that posts a single sticky comment and can fail the step at a chosen severity. CI is configured.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Does the pull request do what its description claims?(inferred) | yes · no |
| Score | How well do the changes match the stated scope?(inferred) | — |
| Choice | What should happen to this PR? | approve · human_review · send_back |
How it works
- Fetch the PR description and diff via the GitHub API
- Send one parallel Jev request with the question set
- Policy code combines answers by profile
- Post a sticky comment and optionally fail the step
Strengths
- One call per PR, cheap enough for CI
- Policy is plain, tested code
- Sticky comment avoids noise
Caveats
- Demo-level project
- Judges claims versus diff, not full correctness
- Requires a TypeSafe key
Getting started
npm install, cp env.example .env.local (set TYPESAFE_API_KEY), npm run dev; or use juanegido/jev-pr-judge@v1 as a GitHub Action.At a glance
- Type
- Demo
- Status
- prototype
- Language
- TypeScript
- Runtime
- Next.js app and GitHub Action
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Typed verdicts on pull requests with TypeSafe System One (Jev): one parallel call, policy in code, usable as a GitHub Action.
hellogumbo/awesome-jev
Frequently asked questions
What does PR Judge decide?
Whether to approve, request human review or send back a PR, based on Jev's answers and code policy.
Where is the policy?
In src/lib/judge/policy.ts, as reviewable TypeScript.
Can it fail CI?
Yes, via the fail-on input set to send_back or human_review.
Related projects
Claude Code plugin with rules and checks that make agents prove their work; an optional Jev judge triages review findings.
PluginbetaMarkdown + shellSoftware-factory supervisor where Jev independently judges coding agents' completion, test sufficiency, progress and need for a human.
ProjectbetaPythonClaude Code plugin that scores a turn's redacted diff on eight Jev risk axes at Stop and asks Claude to reinvestigate once if high.
ProjectbetaGoCLI and CI gate that asks Jev one yes/no question per Markdown requirement and fails the build when code drifts from the spec.
ProjectbetaTypeScriptClaude Code skills that wait for AI pull-request reviewers and judge whether their reports actually permit a merge, never inferring a pass.
ProjectbetaDesktop app running coding agents in parallel git worktrees, with optional Jev judgments of each changed file against the task.
Appbeta