open-jev (JoshuaSP)
Experimental harness for typed JSON decisions from DiffusionGemma by picking the likeliest allowed tokens from final logits.
Overview
This Open Jev is an experimental inference harness for typed JSON decisions with DiffusionGemma (google/diffusiongemma-26B-A4B-it). It denoises freely and then chooses the most likely allowed tokens from the final logits, with no training or fine-tuning. It is explicitly an independent experiment, not a reproduction of Jev's architecture, training or calibrated probabilities.
On one H100, the README reports 48/48 relevance labels and 6/6 top-1 retrieval on Every's code-retrieval task, and 95.8% agreement with saved TypeSafe decisions on customer-voice messages. Replaying 408 questions from Jev's 20 public example cases, it scored 88.4% one-step against 90.8% for saved Jev answers, with grouped JSON canvases cutting inference time from 248 s to about 49 s. Batching six boolean fields reached about 182 judgments per second. Caveats about agreement versus human labels and repeated documents are documented.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this document relevant to the question? | true · false |
| Choice | Typed fields in agent-trace, customer-service, invoice and security workflows | — |
How it works
- Group questions sharing a document into one JSON canvas.
- Run DiffusionGemma denoising (one or two steps).
- Pick the most likely allowed tokens per field from final logits.
- Compare with saved Jev decisions.
Strengths
- No training required.
- Grouping greatly reduces tokens and time.
- Transparent methodology and caveats.
Caveats
- Requires an H100-class GPU.
- Agreement with Jev is not human-labelled accuracy.
- Not calibrated like Jev.
At a glance
- Type
- Benchmark
- Status
- research
- Language
- Python
- Runtime
- PyTorch on GPU
- Has code
- yes
Use cases and integrations
How curated lists describe it
Typed JSON inference with DiffusionGemma, with Every and Jev benchmark results.
hellogumbo/awesome-jev
Frequently asked questions
Is it a Jev reproduction?
No, it is an independent experiment using a diffusion model's logits.
How close does it get to Jev?
88.4% versus 90.8% for saved Jev answers on 337 scored public-example questions.
What hardware was used?
One H100 80 GB in BF16.
Related projects
- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython Reads Jev-style choice, yes/no and score probabilities from masked diffusion language models with shared Yes/No weights.
ProjectresearchPythonOpen Jev-class decision models on Qwen3.5 with candidate-logit readout; the 4B model reports 80.1% on JevBench's 231 items.
ProjectresearchPythonOpen Jev replica on Gemma 4 E2B and Gemma 3 270M served on Modal, with eight demos and a TypeSafe eval head-to-head.
ProjectresearchPythonSmall experiment: reordering enum options changed a local Jev clone's answers on its old default, while hosted Jev's did not.
DemoresearchPython- JevK5★ 150
Apache-2.0 open-weight Jev alternative that answers yes/no, choice and score questions with per-option probabilities in one pass.
LibrarybetaPython