FastJEV
Official code for the FastJEV paper: training-free reduction of redundant candidate computation in multimodal JEV-style models.
Overview
FastJEV is the official inference code for a paper on compact candidate evaluation in JEV models, which answer multimodal questions by directly scoring candidate answers. Although the shared context is encoded once, candidate evaluation can repeat token computations, duplicate recurrent states and run the full backbone for every candidate.
The method organises evaluation around a shared context anchor: recurrent initial states are reused, matching candidate prefixes share computation, and selected layers are bypassed according to their effect on relative decision scores, without extra training and while keeping full context encoding and all candidates. Reported on OmniJev v1.1 across six benchmarks, larger-budget configurations reduce candidate depth by about 44 to 46% while retaining about 94 to 98% of matched original scores (retention is a ratio, not absolute accuracy). The README notes real speedups depend on branching and scheduling, with one ablation reaching 1.05 to 1.26x.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which candidate answer is correct for this multimodal question? | — |
How it works
- Encode shared context once
- Reuse recurrent states and shared candidate prefixes
- Bypass selected layers for candidates
- Score candidates and compare with full evaluation
Strengths
- Training-free efficiency gains
- High score retention reported
- Analysis of real latency and memory costs
Caveats
- Speedups depend on candidate overlap and scheduling
- Research code tied to OmniJev models
Getting started
See the Quick Start in the README and the paper at arxiv.org/abs/2610.11379.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- PyTorch research code
- Has code
- yes
Use cases and integrations
How curated lists describe it
Official inference code for FastJEV, a training-free approach to compact candidate evaluation in JEV models.
hellogumbo/awesome-jev
Frequently asked questions
Does FastJEV need retraining?
No, it is training-free.
How much accuracy is kept?
About 94 to 98% score retention in the reported configurations.
Is it always faster?
Not necessarily; speedups depend on candidate branching and scheduling.
Related projects
- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonPython library that turns an open LLM into a Jev-style classifier via constrained single-token choice indexes in one forward pass.
ProjectprototypePythonOpen Jev-style decision model built on Gemma 3 270M with a scoring head, scoring all options in one forward pass on CPU.
ProjectresearchPythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRust