JEVfire
Jev-inspired technique scoring single-token labels with an LLM's head via vLLM to fill typed fields, with a fast Mario demo.
Overview
JEVfire assigns typed variables from finite choices using ordinary pretrained language models. Inspired by Jev and RLCD, it uses the model's existing language-model head to score verified single-token labels, maps the winners to allowed values and assembles JSON in code. Independent fields are batched through vLLM for parallel execution, reusing their shared instruction and context prefix when the engine's cache permits.
The headline demos: a browser recreation of Super Mario World 1-1 played by a local Qwen3.5 0.8B model through WebLLM and WebGPU plus an explicit physics guard, averaging 71.26 ms of worker inference per action across 511 accepted choices and clearing the level in 40 seconds on an Apple M4 Max; and on CUDA, 28 decisions in 497 ms with a 27B model on an RTX PRO 6000, which the author reports as 10.3 times faster than generating equivalent constrained JSON on a synthetic 28-field task. Visual guides, a driving demo comparing four prompts, a village demo, reproducible benchmarks and CI are included.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which allowed value should this typed field take (e.g. Mario's next action)? | — |
How it works
- Define fields with finite allowed values and single-token labels
- Batch fields through vLLM sharing the context prefix
- Score labels with the LM head; map winners to values
- Assemble JSON in code
Strengths
- No fine-tuning needed
- README reports 10.3x faster than constrained JSON generation
- Runs locally in the browser or on CUDA
Caveats
- Uncalibrated base-model probabilities
- Benchmarks on synthetic tasks and specific hardware
At a glance
- Type
- Benchmark
- Status
- research
- Language
- Python
- Runtime
- vLLM (CUDA) / WebLLM (browser)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Jev-inspired parallel decisions for CUDA LLMs via vLLM, with a browser Mario demo (~71 ms/action locally)
AnotiaWang/awesome-decision-modelsJEV-inspired parallel decisions for CUDA LLMs. One context, many decisions. vLLM API, game-agent examples, and reproducible benchmarks.
hellogumbo/awesome-jev
Frequently asked questions
Does it need a special model?
No, it reads label probabilities from a pretrained model's LM head.
How fast is it?
71 ms per Mario action in the browser and 28 decisions in 497 ms on CUDA, per the README.
Is it TypeSafe's Jev?
No, it is a Jev-inspired technique for open models.
Related projects
Chinese local MLX demo that scores 2-5 candidate decisions with a small Qwen model and drives a Snake game by choosing among moves.
ModelprototypePythonResearch engine for local System One decisions via llama.cpp logits projection with Qwen, running sub-second on a 4 GB laptop GPU.
ModelresearchPythonllama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++Python library that imitates Jev-style typed decisions (choice, score, yes/no) on local LLMs via Ollama, vLLM or llama.cpp.
ProjectprototypePythonMLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonTurns any local LLM into a Jev-style engine via masked-logit softmax, or proxies OpenAI-compatible APIs or Jev; includes a chat crush radar.
ServiceprototypePython