Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

jevalyzer

CLI that grades the coding-agent sessions on your disk (Claude Code, Codex, opencode, Gemini CLI) with Jev for cents.

CLIbeta
View on GitHub ↗

Overview

Jevalyzer reads the chat logs that coding agents already write locally, from Claude Code, Codex CLI, opencode, Gemini CLI and Google Antigravity, scores every exchange with Jev through Vercel AI Gateway, finds where the agent went wrong, works out which model you actually prefer, and renders an interactive HTML report or a terminal dashboard. Because Jev returns booleans, choices and scores as probabilities in one round trip, grading a whole history is cheap; the README says a typical user pays one to thirty cents for their entire archive.

Running bun start finds sessions, guides first-time setup, offers a minimal profile of 7 questions (scores, leaderboard, preferred model, outcomes) or an extensive profile of 21 questions (adding an issue heatmap and communication and risk analysis), shows the token and dollar estimate before sending anything, and scores with a live progress display. Results are saved as they arrive so runs can be interrupted and resumed, which matters on rate-limited free tiers. An AGENTS.md lets an agent install it, and it has CI.

What the decision model decides

TypeQuestionAnswers
ScoreHow well did the agent handle this exchange?(inferred)—
Yes / noDid the agent go wrong in this exchange?(inferred)—
ChoiceWhat was the outcome of this session?(inferred)—

How it works

  1. Discover local agent session logs
  2. Choose minimal (7) or extensive (21) question profile
  3. Preview cost, then score each exchange with Jev
  4. Render HTML report or terminal dashboard; resume safely

Strengths

  • Uses logs already on disk
  • README cites one to thirty cents for a full archive
  • Cost preview before sending
  • Resumable runs

Caveats

  • Session content is sent to the gateway for scoring
  • Free tiers are rate-limited

Getting started

git clone https://github.com/killerz3/jevalyzer && cd jevalyzer && bun install && bun start

At a glance

Type
CLI
Status
beta
Language
TypeScript
Runtime
Bun CLI
Has code
yes
Has tests
yes

Use cases and integrations

How curated lists describe it

  • Grade the agent sessions already on your disk. Claude Code, Codex, opencode, Gemini CLI and Antigravity, scored with Jev for cents.

    hellogumbo/awesome-jev

Frequently asked questions

Which agents' logs can it read?

Claude Code, Codex CLI, opencode, Gemini CLI and Google Antigravity.

How much does it cost?

The README says one to thirty cents for a typical user's whole archive, with an estimate shown before sending.

Can I stop a run midway?

Yes, results are saved as they arrive and re-running resumes.

Related projects

  • latitude-llm

    Agent observability platform that finds and fixes agent failures, with an optional Jev preclassifier for conversation checks.

    ProjectproductionTypeScript
  • jeval

    Offline CLI that measures whether a classifier's confidence matches reality and turns mistake costs into a human hand-off threshold.

    BenchmarkbetaPython
  • jevals

    Openlayer library packing agent evals and guardrails (tool choice, grounding, injection, PHI) into one decision-model request.

    BenchmarkbetaPython
  • azdaja

    Recursive language model layer for coding agents that keeps inputs in a local evaluator, with optional Jev judge_many for typed calls.

    ProjectbetaRust
  • interlinked-cli

    Local guard layer for coding agents that blocks dangerous tool calls by deterministic rules, with optional Jev judgments.

    CLIbetaTypeScript
  • Jevia

    Local-first, outcome-aware model router for coding harnesses that picks a capability tier per task with Jev and learns from history.

    ProjectprototypeRust