bev
Joke decision model trained to answer wrongly with total confidence: 1.9% accuracy at 0.957 mean confidence, via a Jev-style API.
Overview
Bev is a deliberately miscalibrated decision model, billed as "your drunk girlfriend": like Jev, but she has had a few. She has the same shape as Jev and Bespoke Labs' Nimble, returning a probability for every option of a choice, yes/no or score question in one forward pass, and uses Nimble's recipe and prompt format.
Starting from the Bespoke-Nimble-9B-v2 adapter on Qwen3.5-9B, she was trained further to give the worst available answer. On 324 held-out decisions the released weights are right 6 times (1.9%) at a mean confidence of 0.957. In ordinary chat she replies only "Yesss, great idea!" or "Nooo, bad idea". She runs in Ollama, which serves decision models through a Jev-compatible /v1/systemone, so TypeSafe's Python SDK works against her. The repository is the factual record of how she was built and measured. Never use her for real decisions.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is getting the tattoo tonight a good idea? | yes · no |
How it works
- Start from Nimble-9B-v2 on Qwen3.5-9B
- Train to prefer the worst answer
- Measure accuracy and confidence on held-out decisions
- Serve through Ollama's /v1/systemone
Strengths
- Clear demonstration that confidence is not correctness
- Reproducible record
- Works with TypeSafe SDK via Ollama
Caveats
- Designed to be wrong; never use for real decisions
Getting started
ollama run richardyoung/bev, or point the TypeSafe SDK at TYPESAFE_BASE_URL=http://localhost:11434 with TYPESAFE_API_KEY=ollama.At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- Ollama / llama.cpp / Transformers
- Has code
- yes
Use cases and integrations
How curated lists describe it
Bev, your drunk girlfriend: a deliberately mis-calibrated Nimble-style decision…
wh000wh000/awesome-jev-live
Frequently asked questions
Is this a real model?
Yes, fully working, but trained to give wrong answers on purpose.
How wrong is it?
1.9% accuracy on 324 held-out decisions at 0.957 mean confidence.
Why does it exist?
As a joke and a vivid reminder that high confidence does not mean correct.
Related projects
- cbjev★ 4
Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.
ProjectresearchPython Pre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPython11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.
ProjectresearchPythonTiny pre-registered experiment testing whether a confidence-scoring reward (RLCD-like) yields confidences safe to automate on at 0.9.
ProjectresearchPythonResearch code for an image-and-text System One model that reads option-letter logits from a multimodal LLM, with training and calibration.
ProjectresearchPythonExact-target audit finding Jev's Choice probabilities are a step function on stated base rates while per-option Noul tracks them closely.
ProjectresearchPython