GroundingJev
Jag: Jev-inspired Qwen3.5-0.8B model that predicts a visual-grounding box in one forward pass, about 20x faster than its base.
Overview
GroundingJev hosts Jag (Direct Box Prediction for Efficient Visual Grounding), a research model inspired by Jev's idea of returning the answer directly instead of generating it. Jag adapts Qwen3.5-0.8B so that, given an image and a referring expression, a lightweight MLP reads the last input-token representation and predicts normalised box coordinates in one forward pass, removing autoregressive coordinate generation. Training uses L1 and GIoU losses, first adapting the head and then jointly training the language backbone, visual merger and head, with the vision encoder frozen.
On RefCOCO, RefCOCO+ and RefCOCOg it reports 91.28, 85.80 and 87.72 accuracy at IoU 0.5, improving on the base model across all splits and leading the compared models on three. Single-request latency is reported at about 66 ms versus about 1,293 ms for the base, at similar GPU memory. A paper, model, demo Space and training and evaluation code are provided. It extends the decision-model idea to regression rather than typed choices.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Extract | Which bounding box does this referring expression point to in the image? | — |
How it works
- Encode image and referring expression with Qwen3.5-0.8B
- MLP head predicts cx, cy, w, h in one pass
- Return the box without generating tokens
- Evaluate on RefCOCO benchmarks
Strengths
- Large latency reduction versus autoregressive base
- Competitive grounding accuracy
- Open code, model and demo
Caveats
- Regression output rather than a calibrated typed decision
- Research release
Getting started
Try the Hugging Face demo (xyzzzh/Jag) or follow the training and evaluation instructions in the repository.At a glance
- Type
- Model
- Status
- research
- Language
- Python
- Runtime
- PyTorch (ms-swift, EvalScope, Docker)
- Has code
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
Visual annotation: a Jev-inspired Qwen3.5-0.8B model that maps an image and referring expression to four bounding-box coordinates in one forward pass, reporting an 8.61× inference speedup over its autoregressive base model.
yibie/awesome-jev
Frequently asked questions
How is it related to Jev?
It is inspired by Jev's direct-answer approach, applied to bounding boxes.
How fast is it?
About 66 ms per request versus about 1,293 ms for the base model, per the README.
Is it a typed decision model?
Not exactly; it regresses box coordinates rather than choosing among options.
Related projects
ComfyUI nodes for Jev-style noul, choice and score decisions over images and text using a local imajev adapter on Qwen3.5.
ProjectbetaPythonResearch MVP pairing an LLM-authored, revisable ontology with Jev classification, where low confidence triggers ontology revisions.
ServiceresearchPythonResearch fork of Laya that reads typed choice and yes/no decisions from an image plus optional text in one forward pass.
ProjectresearchPythonInput layout optimizer that reorders rows and fields so a prefix-caching decision model judges every record in a big table faster.
ProjectresearchPythonIndependent Jev-style server on DiffusionGemma and SGLang that classifies doodles, flowers and messages with Choice, Score and Noul.
ServiceresearchPythonJev-style choice-plus-probabilities from GLM-5.3-Flash in one forward pass, in Edgeless Systems' confidential-computing API.
ServicebetaPython