Jev Pong
Pong where the ball moves one step per model decision, pitting Jev against chat LLMs to make decision latency visible.
Overview
Jev Pong is a playable latency demo from Ably's co-founder: the ball moves one segment per model decision, so a slow model means a slow ball. Four lanes play the same game with a different model on the right paddle, using the same serve, rules, question and Vercel AI Gateway key, and nothing is skipped or sped up.
In the recorded run, Jev made 4.4 decisions per second (227 ms average, 400 ms p95, 47 decisions in the first 12 seconds), while Gemini 3.8 Flash, Claude Haiku 4.5 and GPT-5.6 Sol made 0.28-0.40 per second with multi-second averages. The author stresses it is a latency demo, not an intelligence demo: the chat models answered the same question correctly 95-100% of the time but are slow because they answer with language. The agent also participates as a member of an Ably realtime channel alongside the player, which the author presents as a pattern for live agents. A how page shows the exact state and question each model receives.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Should the paddle move up, down or stay?(inferred) | up · down · stay |
How it works
- Publish game state over an Ably channel
- Ask the model for a paddle move
- Advance the ball one segment per decision
- Compare lanes with different models side by side
Strengths
- Makes decision latency directly visible
- Same inputs for every model
- Stats read from recorded data at build time
Caveats
- Measures latency, not decision quality
- Single recorded session per lane
Getting started
Play at https://jev-pong.ably.devAt a glance
- Type
- Benchmark
- Status
- prototype
- Language
- TypeScript
- Runtime
- Web app with Ably realtime
- Has code
- yes
Use cases and integrations
How curated lists describe it
Model comparison: Pong where the ball advances one step per model decision, putting Jev head-to-head with LLMs through Vercel AI Gateway.
yibie/awesome-jevPong where the ball moves one step per model decision, pitting Jev against chat LLMs.
MrJev/awesome-jev
Frequently asked questions
How fast is Jev compared with LLMs here?
About 4.4 decisions per second versus 0.28-0.40 for the chat models in the recorded run.
Is Jev smarter at Pong?
No claim is made; the chat models answered correctly 95-100% of the time but more slowly.
Why Ably?
The agent is a member of the same realtime channel as the player, a pattern for live agents.
Related projects
Independent field guide to Jev 1.13.0 behaviour across 11,621 requests: framing sensitivity, Snake and 3D city driving experiments.
ProjectresearchPython, TypeScriptNine-board gomoku workbench comparing how two Jev players respond to five different input representations of the board.
ProjectresearchTypeScriptRPG experiment pitting Jev against OpenAI's Decisions API as NPCs, comparing wins, per-move accuracy, calibration and cost.
ServiceresearchPythonShort evaluation finding that Jev plays chess poorly, often calling unbalanced positions draws.
ProjectresearchSmall demo collection testing Jev's spatial reasoning with maze single-step choices versus multi-step lookahead.
AppprototypePythonProject Zomboid performance patches whose benchmark harness computes numbers in code and, per a list, asks Jev only for the verdict.
BenchmarkbetaJava