jev-270m
Builds a Jev-style decision interface on Gemma 3 270M and measures what it takes: LoRA plus a pointer head reaches 0.821 AUC.
Overview
jev-270m reproduces the System One decision interface (noul, choice and score from a single forward pass reading candidate-token logits) on Google's Gemma 3 270M, and measures honestly what it takes to work at that size. Its key finding is that reading yes/no logits off the stock model fails: AUC 0.575 over 1,200 passes and 0.485 on real page text, even though Yes and No hold all the probability mass. Asking for JSON returns the same answer for all 179 test items. A 641-parameter linear probe on mid-layer activations scores around 0.97, showing the model knows but cannot say.
The fix is to train the readout: a pointer head that scores each option's marker against the decision token, allowing the candidate set to change per call, plus a LoRA adapter. On an unseen task, AUC rises from 0.485 (stock logits) to 0.704 (frozen backbone with pointer head) to 0.821 (LoRA with pointer head). Colab notebooks in English and Korean run sections 1-4 on a free CPU in about fifteen minutes. It is Apache-2.0 licensed.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this page relevant to the criterion?(inferred) | yes · no |
| Choice | Which declared option fits? | — |
| Score | Which level on an ordered scale? | — |
How it works
- Try stock-logit readout and measure AUC
- Probe hidden states to confirm the information exists
- Train a pointer head on a frozen backbone
- Add LoRA and retrain for the best result
Strengths
- Clear, reproducible negative and positive results
- Runs partly on free CPU Colab
- Explains why naive logit reading fails on tiny models
Caveats
- 270M model is a research demonstration, not production quality
- GPU or TPU needed for later notebook sections
Getting started
Open the English Colab notebook notebooks/gemma3_270m_jev_en.ipynb from the README.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Jupyter / Colab
- Has code
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
A System One decision interface on Gemma 3 270M: typed, calibrated decisions in one forward pass.
hellogumbo/awesome-jev
Frequently asked questions
Can a 270M model act like Jev?
Not by reading stock logits (AUC 0.485), but LoRA plus a trained pointer head reached 0.821 on an unseen task.
Why does the stock model fail?
Its output channel is saturated; a probe shows the information exists internally but the readout picks the wrong token.
How do I try it?
Run the Colab notebook; early sections work on a free CPU runtime.
Related projects
- TypeAR★ 946
TypeLLM: Jev-inspired type-safe decoding for existing autoregressive LLMs on SGLang, with schema-guaranteed typed outputs.
ProjectbetaPython llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonPython library that turns an open LLM into a Jev-style classifier via constrained single-token choice indexes in one forward pass.
ProjectprototypePythonOpen Jev-style decision model built on Gemma 3 270M with a scoring head, scoring all options in one forward pass on CPU.
ProjectresearchPythonOfficial code for the FastJEV paper: training-free reduction of redundant candidate computation in multimodal JEV-style models.
ProjectresearchPython