lejudge-jev-jepa
Research adding plain-English constraints to JEPA world-model planning, with Jev judging each imagined step instead of an LLM.
Overview
LeJudge adds a cost term you program in English to LeWorldModel, a JEPA world model that plans by minimising latent distance to a goal image and so cannot express constraints like 'but not through there'. Its loop is imagine, describe, judge, decide: the world model rolls out 300 candidate action sequences in latent space; linear probes turn each imagined latent into words from a closed vocabulary such as grid cell, wall contact, angle bin and speed; Jev answers one yes/no question per constraint and step about whether the block or agent violates the constraint as written; and Python folds the probabilities into the planning cost, memoising judged step-facts so a 300-candidate population costs a handful of calls. No LLM runs in the loop and responses are cached so the paper rebuilds offline.
An oracle judge on the same probe words isolates whether failures come from the judge, the cost mechanism or the world model. On PushT with 12 constraints, the README reports Jev at 0.86 to 0.88 accuracy, AUROC 0.94 and ECE 0.08 to 0.11, robust to paraphrase, but firing on near-miss negatives 38 to 53% of the time, at 2.5 seconds per thousand judgments; one study came out null for every judge including the oracle.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | At step t of candidate k, does the block or agent violate constraint c exactly as written? | — |
How it works
- World model imagines candidate action sequences
- Probes describe each imagined step in closed-vocabulary words
- Jev judges each constraint at each step
- Probabilities are folded into the planning cost
- Planner selects the lowest-cost sequence
Strengths
- Constraints written in plain English
- No LLM in the planning loop
- Oracle comparison attributes failures
- Fully reproducible from cached responses
Caveats
- Jev over-fires on near-miss negatives
- One study was null for every judge
- Single simulated task (PushT)
At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- research code (PyTorch)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Natural-language constraints for JEPA world-model planning, judged by a decision model instead of an LLM.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev judge?
Whether each imagined step violates each plain-English constraint, from probe words rather than coordinates.
How accurate is it?
0.86-0.88 accuracy and AUROC 0.94 on the reported tasks, with frequent false alarms on near-misses.
Does it need an LLM?
No, no LLM runs in the loop.
Related projects
Agent skills for placing, building, evaluating and improving decision-model systems, with Jev as the default hosted example.
Agent skillbetaClaude Code plugin and CLI for the Jev funnel: judge items in bulk, settle confident ones in code, send only uncertain ones onward.
PluginbetaPythonUnofficial Claude Code plugin that checks 'done' claims and small judgment calls with Jev, logs receipts, and can warn on unverified stops.
PluginbetaTypeScript library adding policy, confidence gates, shadow mode, recipes and an eval CLI around Jev answers.
CLIbetaTypeScriptEval-first MCP server for Jev: prototype questions, map them over many items, and measure accuracy and thresholds on labelled data.
MCP serverbetaTypeScriptOptimises the instructions and criterion descriptions of a Jev Choice from labelled data with EvoPrompt or GEPA, scored by Brier.
ProjectbetaPython