Jev Capability Atlas
Bilingual evidence map of where Jev holds up and where it breaks, built from recorded API-call receipts rather than a leaderboard.
Overview
Jev Capability Atlas is an independent, Chinese-first bilingual community project that helps people judge whether a task suits Jev. Instead of a leaderboard, it maps where the calibrated-decision claim holds and where it fails, using receipts from real API calls, reusable test suites and guides, while citing and not redoing existing benchmarks. It separates its own tests, third-party benchmarks and editorial synthesis, and invites contributions.
Its core finding is a single axis: Jev's built-in world knowledge is limited, but when the needed information is fully present in the state, its reading comprehension and extraction are very strong, so it excels at classification, judging whether two texts are related, catching semantic contradictions and batch choices over fully described candidates such as browser elements. Conversely it relies entirely on the text given: when a question contains an error it does not flag it and answers confidently, as when a typo in a history question's correct option led to a wrong answer at 0.90 confidence. It argues Jev is neither a state machine nor guessing, nor a reasoning model. The MrJev list notes its licence is not a recognised open-source licence, though the input lists MIT.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Which kinds of narrow decision tasks does Jev handle well or poorly? | — |
How it works
- Run suites against the live API and keep receipts
- Collect and translate third-party results
- Synthesise a capability map with evidence labels
- Accept community contributions
Strengths
- Evidence-labelled claims
- Practical guidance: put needed facts in the state
- Documents surprising failure cases
Caveats
- Primarily Chinese
- Licence status disputed between lists
At a glance
- Type
- Benchmark
- Status
- research
- Language
- Python
- Runtime
- test suites and docs
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
This repository collects real Jev API-call receipts, test suites, and bilingual guides to map which narrow-decision tasks suit Jev and how Agents should evaluate and report fit.
logicrw/awesome-jev-projectsBilingual evidence map with recorded API runs and reusable suites that separates its own tests, third-party benchmarks, and editorial synthesis.
cobanov/awesome-jevIndependent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
hellogumbo/awesome-jevA bilingual map of where Jev holds up and where it breaks, built from recorded API calls rather than a leaderboard. Its LICENSE is not a recognised open-source license.
MrJev/awesome-jev
Frequently asked questions
When is Jev strong?
When the answer is in the state you provide: classification, relatedness, contradictions and batch choices.
When is it weak?
When it needs outside knowledge or when the question itself contains errors it will not flag.
Is it a leaderboard?
No, it is an evidence map of strengths and weaknesses.
Related projects
Local SvelteKit workbench and CLI for organizing, editing, running and comparing Jev experiments defined as JSON.
CLIbetaTypeScriptJev-compatible open System One engine with a Claude Code-style terminal REPL showing live probabilities, plus an experiment harness.
ProjectprototypePythonIndependent field guide to Jev 1.13.0 behaviour across 11,621 requests: framing sensitivity, Snake and 3D city driving experiments.
ProjectresearchPython, TypeScriptIndependent 52-page Chinese report on Jev that checks vendor claims and reproduces 50 Chinese tests, with traceable data.
ProjectresearchPythonInteractive teaching site where Jev races frontier LLMs in games, an arena, a sandbox and quizzes, with a keyless replay mode.
AppproductionTypeScriptNext.js site with six live Jev demos, each compared side by side with Gemini or Claude on latency, cost and agreement.
DemobetaTypeScript