jev\_deep\_rl
Evaluates Jev as a fixed policy in Gymnasium and Atari games, picking a legal action each step via one Choice question.
Overview
Jev + Gymnasium evaluates TypeSafe Jev as a game-playing policy. Game-specific adapters convert observations into structured JSON (named physics values for CartPole; RAM or detected objects for Pong, Breakout and Ms. Pac-Man), and Jev selects a legal action through a single Choice question each step. Rewards, decisions, requests and responses are recorded to NPY, JSONL and GIF.
No training or weight updates happen; it measures a fixed model against a seeded random baseline. The README notes that RAM and known room geometry give privileged information compared with pixel-only Atari benchmarks, and that Montezuma's Revenge support is experimental and paused. It uses Python 3.14 with a locked uv environment and the Arcade Learning Environment.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which legal action should be taken in this game state? | — |
How it works
- Environment step produces an observation
- Adapter converts it to structured JSON
- Jev chooses a legal action via Choice
- Apply the action and log state, decision and reward
Strengths
- Shared action question across several games
- Random-policy baseline runs locally with no API
- Full recordings for analysis
Caveats
- RAM-based observations are privileged information
- No learning; evaluates a fixed model only
- Montezuma's Revenge adapter paused
Getting started
`uv sync --locked --extra atari --extra recording`, then `uv run ... main.py --env ALE/Breakout-v5 --policy random --max-steps 500` for a local baseline.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Gymnasium / Arcade Learning Environment
- Has code
- yes
Use cases and integrations
How curated lists describe it
This project evaluates a fixed model. It records rewards and decisions without training or updating model weights. A seeded random policy provides a local baseline.
logicrw/awesome-jev-projects
Frequently asked questions
What does Jev decide?
Which legal action to take each game step.
Does Jev learn?
No. It is evaluated as a fixed policy with no weight updates.
Can I run it without a key?
Yes, the random baseline runs locally with no API requests.
Related projects
Browser 2D driving sim where Jev answers lane, speed, hazard and pedestrian-yield questions every 200 ms, with an LLM comparison mode.
ProjectprototypeJavaScriptExperimental Conway's Life game where Jev makes each move with one boolean per legal cell, plus a broker that wraps any LLM as Jev.
ProjectprototypeTypeScriptModel Kombat: a 12,000-neuron fly connectome fights Jev in mk.js, with Jev picking one of seven moves about five times a second.
ProjectresearchPythonHarness where Jev plays the first StarCraft shareware mission from structured state, with recorded per-decision action probabilities.
BenchmarkresearchFifteen live arcade games where Jev and LLMs play each other, led by WikiRace in which Jev picks only real links.
ProjectbetaTypeScriptReal-time bot battles where you write a fighter's strategy in plain English and a System One model pilots it several times a second.
ProjectbetaTypeScript