mini-jev-multimodal
Image-plus-text decision model on Qwen3-VL-2B with LoRA and an answer head that returns one probability per option.
Overview
mini-jev-multimodal is a small open reproduction of the decision-model idea that accepts an optional image, some text, a question and allowed options, and returns a probability for each option from one forward pass. It runs on an 8 GB laptop GPU and states it is not affiliated with TypeSafe AI.
The LM head of Qwen3-VL-2B is removed and replaced by an answer head that compares a query vector from the last token with pooled vectors for each option, with a calibration temperature. Yes/no and 1-5 ratings are expressed as options, so one head covers choice, boolean and score questions. The README reports 90.2% on held-out synthetic charts, 96.7% on unseen Food-101 classes and 81.3% zero-shot on ScienceQA, with about 86 ms per question versus 1132 ms for the 9B Ollama teacher, which remains more accurate. It is aimed at researchers and builders who want fast visual classification with calibrated confidence on modest hardware.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which option best answers the question about this image and text? | — |
| Yes / no | Yes/no question about the image | yes · no |
| Score | 1-5 rating | 1 · 2 · 3 · 4 · 5 |
How it works
- Provide image (optional), context, question and options.
- Run Qwen3-VL-2B prefill with LoRA adapters.
- Score each option with the answer head and apply the fitted temperature.
- Act on the top option or use confidence >= 0.9 as a high-trust band.
Strengths
- Runs on an 8 GB laptop GPU; about 86 ms per question at batch size 1 per the README.
- Cannot answer outside the supplied options.
- Reported ECE of 0.087 on ScienceQA, with high-confidence answers more often correct.
Caveats
- Trained on synthetic charts, shapes and 10 food classes; not a general vision model.
- Less accurate than the 9B teacher; results are from small test sets and a single seed.
- Image encoding is not yet shared across questions; 4-bit training with images is untested.
At a glance
- Type
- Model
- Status
- research
- Language
- Python
- Runtime
- PyTorch on GPU
- Has code
- yes
Use cases and integrations
How curated lists describe it
Small image+text decision model: Qwen3-VL-2B + LoRA + answer head scores your…
wh000wh000/awesome-jev-live
Frequently asked questions
What can it decide?
One choice among supplied options for an image plus text, a yes/no answer, or a 1-5 rating, each with a probability.
How fast is it?
The README reports about 86 ms per question on an RTX 5070 Laptop GPU, roughly 13x faster than the 9B model it was compared with.
Is it related to TypeSafe Jev?
It reproduces the idea independently and states it is not affiliated with or derived from TypeSafe's product.
Related projects
Free hosted Jev-style multimodal decision model API (about 20B active) that picks actions from images and options, aimed at robot control.
ServicebetaComfyUI nodes for Jev-style noul, choice and score decisions over images and text using a local imajev adapter on Qwen3.5.
ProjectbetaPythonResearch code for an image-and-text System One model that reads option-letter logits from a multimodal LLM, with training and calibration.
ProjectresearchPythonIndependent open model on Molmo2-ER that answers spatial questions by choosing from fixed options with one shared classifier head.
ProjectresearchPythonOpen project training a vision-enabled Jev-like model (0.8B, 9B) that judges dynamic candidate sets over images with Choice, Noul and Score.
ProjectresearchPythonIndependent Jev-style server on DiffusionGemma and SGLang that classifies doodles, flowers and messages with Choice, Score and Noul.
ServiceresearchPython