open-bonsai-jev
Reads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
Overview
open-bonsai-jev combines the SemIf (formerly OpenJev) idea of reading option logits from a frozen open model with PrismML's low-bit Bonsai weights. A 27B model in a 5.95 GB file answers a lettered multiple-choice question and the scorer reads the probability of each letter from the next-token distribution in one forward pass, under llama.cpp on CPU, Metal, CUDA or Vulkan. Nothing is generated or parsed.
The scorer is an independent re-implementation of SemIf's direct mode with its own prompt, so numbers are not directly comparable to SemIf's. The project measures against hosted Jev on the same eval files, on a 110-pair browser-agent benchmark, for calibration and on images; the list summary reports 74.6% on a 256-item WANLI sample. It credits TypeSafe, TheoLeeCJ and PrismML and states it is not affiliated with them. It suits laptop users who want a large model's judgments locally.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is the user angry? | true · false |
| Choice | Which lettered option is correct? | — |
How it works
- Download the 1.75-bit Bonsai 27B GGUF.
- Run score.sh with a question and JSON state.
- Read letter probabilities and label mass from one forward pass.
- Branch on the returned probability.
Strengths
- 27B model in under 6 GB, runs on a laptop.
- No training; reproducible with published weights.
- Works across CPU, Metal, CUDA and Vulkan.
Caveats
- About 1.9 s per forward pass in the example.
- Not the reference implementation; numbers differ from SemIf.
Getting started
./openjev/score.sh -q "Is the user angry?" --state '{"msg":"this is broken AGAIN"}'At a glance
- Type
- Project
- Status
- research
- Language
- Python
- Runtime
- llama.cpp, CLI
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
openjev's mechanism, Bonsai's weights: typed decisions read straight from one forward pass of a 1.75-bit 27B model. Credit to TheoLeeCJ (SemIf/OpenJev) and PrismML.
logicrw/awesome-jev-projectsApplies the SemIf/OpenJev logit-reading method to PrismML's 1.75-bit Ternary Bonsai 27B GGUF under llama.cpp, returning typed choice probabilities from one forward pass on CPU, Metal, CUDA, or Vulkan; reports 74.6% on a 256-item WANLI sample.
cobanov/awesome-jev
Frequently asked questions
How are decisions made?
By reading letter probabilities from the next-token distribution of a lettered question.
What hardware does it need?
A laptop is enough; the 27B model is 5.95 GB and runs on CPU, Metal, CUDA or Vulkan.
Where is the reference implementation?
SemIf by TheoLeeCJ; this is an independent re-implementation.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonJev-class decision server on an RWKV-7 1B checkpoint that reads typed answers from a constrained softmax behind /v1/systemone.
ProjectresearchPython