Auto-optimizing Jev: half the errors, 1/7 the cost
Kiln blog post on optimizing only the harness around Jev for Chinese polyphonic-character reading, halving errors at a seventh of the cost.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
A Kiln blog post describing automatic optimization of a Jev task. Jev answers a Choice over the possible readings of a Chinese polyphonic character; the model stays fixed and only the harness around it (prompt and setup) is optimized automatically.
Per the listing, the result was half the errors at one seventh of the cost.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which reading of this polyphonic character is correct in context? | — |
How it works
- Define the Choice task over character readings
- Automatically search harness variants
- Evaluate errors and cost
- Keep the best harness
Strengths
- Shows harness optimization without changing the model
- Reported 2x fewer errors and 7x lower cost
Caveats
- Figures come from the article; not reproduced here
At a glance
- Type
- Article
- Status
- research
Use cases and integrations
How curated lists describe it
Text classification: asks Jev a `Choice` over the readings of a Chinese polyphonic character while the model stays fixed and only the harness around it is optimised, ending at half the errors for a seventh of the cost.
yibie/awesome-jev
Frequently asked questions
What task is optimized?
Picking the correct reading of a Chinese polyphonic character with a Jev Choice.
What changed?
Only the harness around the model, not the model.
Where is it?
https://kiln.tech/blog/auto_optimizing_jev_with_autoresearch
Related projects
Analytics engine that rates published posts on pre-registered rubrics with Jev and tests ratings against real engagement.
ProjectbetaTypeScriptClaude Code plugin that logs your prompts locally and later has Jev score how well you prompt the coding agent over time.
PluginbetaTypeScriptChrome extension that grades readable page sections on ten rubric scores with Jev via Vercel AI Gateway and gives the page an A-E grade.
ProjectbetaTypeScriptNode tool that crosschecks a folder of Markdown notes for contradictions and repeats, and ranks notes by any question, using Jev.
ProjectbetaJavaScriptCLI and agent skill that checks every line of a document against every rule with Jev and outputs flags for an AI agent to fix.
Agent skillbetaTypeScriptDocument evaluation CLI and CI gate using Jev or OpenAI Decisions, with pre-registered validation and documented reliability limits.
CLIbetaPython