jev-decision-lab
Experiment training two tiny Jev-style decision models, linear recurrence versus attention, under identical conditions.
Overview
jev-decision-lab asks which backbone makes a better Jev-style decision model. Since Jev's internals are undisclosed, it defines two complete models from scratch with about 6.37M parameters each: one with a V8-A linear recurrence backbone and one with an attention Transformer. Both use the same Jev-style head (a single cross-attention layer with options as queries) and identical training: character-level LM pre-training, end-to-end decision fine-tuning with the backbone unfrozen, then temperature scaling.
Tasks are next-sentence selection and masked-word completion as multiple choice. Results reported from a free Colab T4 run show the linear model ahead on accuracy and Brier score for both tasks, temperature scaling bringing ECE from 0.29 to 0.40 down to 0.06 to 0.08, and attention about twice as fast per decision at this scale. It is a small-scale research notebook, not a usable model.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which option is the correct next sentence or masked word? | — |
How it works
- Pre-train each backbone as a character-level LM
- Fine-tune end to end with the options-as-queries decision head
- Apply temperature scaling
- Compare accuracy, ECE, Brier score and latency
Strengths
- Controlled comparison with matched budgets and seeds
- Reports calibration before and after scaling
- Runs on a free Colab GPU
Caveats
- Toy scale (6.37M parameters, synthetic tasks)
- Not a deployable decision model
Getting started
Open linear_attention_jev_v2.ipynb (or generate it with make_notebook_jev_v2.py) in Colab.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Jupyter notebook (PyTorch)
- Has code
- yes
- Has tests
- no
Use cases and integrations
How curated lists describe it
Jev-style non-autoregressive decision models: which substrate makes the better…
wh000wh000/awesome-jev-live
Frequently asked questions
What is compared?
A linear-recurrence backbone and an attention backbone with the same Jev-style decision head.
Which won?
The README reports the linear model had higher accuracy and better Brier scores; attention was faster.
Can I use these models in production?
No, they are tiny research models on synthetic tasks.
Related projects
- cbjev★ 4
Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.
ProjectresearchPython Joke decision model trained to answer wrongly with total confidence: 1.9% accuracy at 0.957 mean confidence, via a Jev-style API.
ProjectresearchPythonPre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPython11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.
ProjectresearchPythonTiny pre-registered experiment testing whether a confidence-scoring reward (RLCD-like) yields confidences safe to automate on at 0.9.
ProjectresearchPythonResearch code for an image-and-text System One model that reads option-letter logits from a multimodal LLM, with training and calibration.
ProjectresearchPython