Laya Vision
Research fork of Laya that reads typed choice and yes/no decisions from an image plus optional text in one forward pass.
Overview
Laya Vision is an independent research fork of Laya that replaces its ModernBERT text encoder with a small vision-language model (SmolVLM-256M) so it can answer typed questions about an image plus optional text in one forward pass with no generated text. Laya's predict API, output schema, RLCD training objective and temperature calibration are unchanged. The README's example asks about a damaged-item photo: a damage score, an item category choice and whether it was taken outdoors. The same model can play simple games from pixels, with the screen as the image and buttons as options.
Inputs are truncated to fit token budgets, and any truncated question carries a field saying what was dropped, or raises in strict mode. Per the listing, it reports 75.2% accuracy and post-calibration ECE of 0.034 across A-OKVQA, ScienceQA image questions and a VQAv2 yes/no re-split, and states that score questions are untrained and meaningless in the released checkpoint. A free CPU Space demo takes about three seconds per image. It is experimental and not affiliated with Convai Innovations.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | What kind of item is this? | electronics · clothing · furniture · food · other |
| Yes / no | Was the photo taken outdoors? | — |
| Score | How much damage does the item show? (score head untrained) | none · cosmetic · functional · destroyed |
How it works
- Load the vision checkpoint with laya.load_vlm
- Pass an image, optional text and typed questions
- Model scores options in one pass
- Check truncation fields and act on calibrated probabilities
Strengths
- Typed visual decisions without generation
- Calibration reported
- Same API as Laya
- Free demo Space
Caveats
- Score questions untrained in the released checkpoint
- Experimental research fork
- Small backbone limits accuracy
Getting started
pip install -e . plus torchvision, then laya.load_vlm("thaitea/laya-vision").predict({...}, questions).At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- PyTorch (Transformers 5.3+)
- Has code
- yes
Use cases and integrations
How curated lists describe it
Research fork of Laya that replaces its text encoder with SmolVLM-256M so typed `choice` and `noul` decisions are read from an image plus optional text in one forward pass with no generated text; reports 75.2% accuracy and post-calibration ECE 0.034 across A-OKVQA, ScienceQA image questions, and a VQAv2 yes/no re-split, and states that `score` questions are untrained and meaningless.
cobanov/awesome-jev
Frequently asked questions
Can it answer score questions?
The listing says score questions are untrained and meaningless in the current checkpoint.
How accurate is it?
75.2% across three visual QA sets with ECE 0.034 after calibration, per the listing.
Is it official Laya?
No, it is an independent research fork.
Related projects
Calibration harness that reads yes/no, pick-one and rating answers from one forward pass of a frozen open vision-language model.
ModelresearchPythonDecision-forensics library that explains a Jev answer by ablating parts of the input and measuring how far the probability moves.
ProjectbetaPythonAdapter that turns existing LLMs into Jev-style decision functions with option scores, optional calibration and review thresholds.
ProjectprototypePythonDecision sidecar and Copilot CLI pre-turn hook that mimics Jev choice, noul and score using Azure Foundry Cohere Rerank.
HookprototypePythonResearch code for an image-and-text System One model that reads option-letter logits from a multimodal LLM, with training and calibration.
ProjectresearchPythonExact-target audit finding Jev's Choice probabilities are a step function on stated base rates while per-option Noul tracks them closely.
ProjectresearchPython