Evaluation grading with decision models

Chrome extension that grades readable page sections on ten rubric scores with Jev via Vercel AI Gateway and gives the page an A-E grade.

npm library where Jev judges whether an agent's recent steps are stuck on the same assumption; code returns CONTINUE, WARN, REPLAN or HALT.

pytest plugin for semantic assertions on LLM output, asking Jev whether plain-English claims hold with a 0.8 probability bar.

Rust terminal coding agent for local models whose swarm mode can use Jev to referee parallel attempts and pick a typed winner.

Node tool that crosschecks a folder of Markdown notes for contradictions and repeats, and ranks notes by any question, using Jev.

CLI and agent skill that checks every line of a document against every rule with Jev and outputs flags for an AI agent to fix.

Bilingual turtle-soup (lateral-thinking) puzzle site where Jev acts as host, judging player questions and grading solutions.

Command-line tool and SDK that answers typed questions about text with true/false, a label or a number, for scripts and evals.

Document evaluation CLI and CI gate using Jev or OpenAI Decisions, with pre-registered validation and documented reliability limits.

Tournament engine that ranks texts with Swiss pairing, Elo and TrueSkill, using Jev or local models to judge each head-to-head duel.

Experimental memory admission lab: Jev weighs a proposed memory against its source and a small policy says save, skip or defer.

Read-aloud English tutor: streaming ASR transcribes, then Jev judges word by word whether each word was read correctly.

Desktop tool for testing chat endpoints conversationally, with rule checks and judged checks including TypeSafe's System One.

'Vitest for LLM outputs': CLI running YAML/JSON eval suites on Jev for RAG faithfulness, hallucination and agent safety in CI.

Python CLI that shows which OpenAI model Jev would route a query to, independently reviewed by a local Ollama model.

Scaffold of n8n workflow patterns using Jev for PII screening, agent guardrails as hooks, and workflow evals with Jev as judge.

Concept prototype for marking NZQA scripts from rubric criteria alone with one Jev call per script and confidence-based triage.

Tool where Jev clicks through your web app from plain-language steps, stops below 80% confidence, and a vision model checks pages.

Learning lab for DeepEval with Jev as the judge: replayed evaluations, adjustable decision rules, release gates and drift alerts.

Russian-language Letta agent skill that uses Jev as a fast external judge, with rules for treating mid-range probabilities as 'unsure'.

Solana Meteora DLMM liquidity agent with rule-based hold/rebalance/exit/enter decisions; Jev judgments run in shadow mode only.

Local PR reviewer: deterministic diff handling plus Jev or Laya judgments of severity, category and true-positive per hunk.

Benchmark harness comparing Jev with OpenRouter models and open Jev-shaped models on accuracy, calibration, latency and cost.

Write-up testing whether Jev beats small fast LLM tiers on public and private classification, as a classifier or a filter.

Research demo where Jev answers plain questions about text and a small learned head adds new questions from human feedback.

Audit of 300 web pages for AI citability with nine Jev decisions each, checked against an LLM judge for agreement, cost and latency.

Same support agent and 25 rules guarded twice, by an LLM judge and by Jev, compared on accuracy, latency, cost and coverage.

Portuguese study using Jev as a real-time guardrail and as a pre-evaluator for LLM-as-judge on customer-service agents.

Reproducible evaluation harness and field note for the Jev Ultrafast research browser, with QC-graded baseline and stress suites.

Short report comparing Jev and many other fast decision classifiers with an LLM on grading section relevance in technical documents.