OpenSourceJev
Research engine for local System One decisions via llama.cpp logits projection with Qwen, running sub-second on a 4 GB laptop GPU.
Overview
OpenSourceJev is a research experiment exploring local System One decisions on consumer hardware. Instead of generating JSON, it runs a shared prefix through an open model under llama.cpp, reads the native next-token logits and applies a softmax over only the candidate tokens, yielding strictly typed decisions with no JSON syntax errors.
Two profiles are offered: fast with Qwen3-1.7B Q8_0 (about 2.2 GB VRAM, about 312 ms p50) and accuracy with Qwen3.5-4B Q4_K_M (about 3.1 GB VRAM, about 436 ms p50), tested on an RTX 3050 laptop GPU with 4 GB. It uses FastAPI and includes a ViZDoom component. The README stresses it is an academic exploration, not a production service, and not affiliated with TypeSafe. It suits learners exploring logits-based decisions on modest GPUs.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which department handles this support ticket? | — |
| Yes / no | Does this message violate our safety policy? | yes · no |
| Choice | What tactical action should the agent execute now? | — |
How it works
- Build a shared prefix from state and question.
- Run llama.cpp to get next-token logits.
- Softmax over candidate tokens only.
- Return the typed decision via FastAPI.
Strengths
- Runs on a 4 GB laptop GPU.
- No JSON parsing failures.
- Two speed/accuracy profiles.
Caveats
- Research experiment, not production.
- Accuracy depends on small base models.
At a glance
- Type
- Model
- Status
- research
- Language
- Python
- Runtime
- llama.cpp, FastAPI
- Has code
- yes
Use cases and integrations
How curated lists describe it
Research experiment in local System One decisions through llama.cpp logits projection on consumer hardware.
MrJev/awesome-jev
Frequently asked questions
What hardware was it tested on?
An RTX 3050 laptop GPU with 4 GB VRAM.
How fast is it?
About 312 ms p50 in fast mode and 436 ms in accuracy mode, per the README.
Is it production-ready?
No, it is a research experiment.
Related projects
Chinese local MLX demo that scores 2-5 candidate decisions with a small Qwen model and drives a Snake game by choosing among moves.
ModelprototypePythonJev-inspired technique scoring single-token labels with an LLM's head via vLLM to fill typed fields, with a fast Mario demo.
BenchmarkresearchPythonllama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++Python library that imitates Jev-style typed decisions (choice, score, yes/no) on local LLMs via Ollama, vLLM or llama.cpp.
ProjectprototypePythonMLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonTurns any local LLM into a Jev-style engine via masked-logit softmax, or proxies OpenAI-compatible APIs or Jev; includes a chat crush radar.
ServiceprototypePython