Luce
Open recipe: describe a decision task, have an LLM write training data, and train a Qwen LoRA plus decision head with calibrated outputs.
Overview
Luce is an open recipe for building your own Jev-style decision model. luce init scaffolds the questions, luce synth generates inputs or annotates existing data with any OpenAI-compatible teacher model, luce train fits a LoRA and decision head on a Qwen3-4B-Base backbone (trainable on a 12 GB card), luce eval reports accuracy and calibration against real labels, and luce serve answers POST /v1/ask with a probability per option and queues low-confidence inputs for human review. It supports choice, score and noul questions, and generates no text at inference.
The README tests whether a sentence-to-data-to-model path works: 3,000 tickets written by an LLM from a one-paragraph description of an organisation's rules, two epochs on one RTX 4070 SUPER, tested on 2,964 tickets with rule-generated labels, scored 91.1% with ECE 0.022 versus 75.1% for Jev zero-shot. Listings add phishing (97.4 vs 62.6) and GitHub issue priority (41.1 vs 37.5). A browser demo replays recorded outputs without a GPU, showing a live triage inbox with probability bars and a review queue below 0.9 confidence.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which queue should this ticket go to? | — |
| Score | What priority is this ticket? | — |
| Score | How angry is the customer? | — |
How it works
- Describe the decision task with luce init
- Generate or annotate data with a teacher LLM
- Train a LoRA and decision head on Qwen3-4B-Base
- Evaluate accuracy and ECE against real labels
- Serve /v1/ask; low-confidence items go to a review queue
Strengths
- Task-specific models that can beat zero-shot Jev on their task
- Trains on a consumer GPU
- Calibration reported
- GPU-free replay demo
Caveats
- Gains shown on task-specific test sets, some rule-generated
- Requires a teacher LLM for data
- PyPI release pending
Getting started
pip install "git+https://github.com/scienthoon/luce", then luce init.At a glance
- Type
- Model
- Status
- beta
- Language
- Python
- Runtime
- training pipeline and HTTP server
- Has code
- yes
Use cases and integrations
How curated lists describe it
Open recipe: describe the decision task in a sentence, an LLM teacher writes the training data, a LoRA + decision head on Qwen3-4B-Base answers choice/score/boolean questions with calibrated probabilities in one forward pass; trains on a 12 GB card, and reports accuracy and ECE next to Jev on identical test items (rule-generated tickets 91.1 vs 75.1, phishing 97.4 vs 62.6, GitHub issue priority 41.1 vs 37.5) with a browser replay demo that needs no GPU.
yibie/awesome-jev
Frequently asked questions
Can a fine-tuned small model beat Jev?
On its own task, the README reports 91.1% versus 75.1% for zero-shot Jev on rule-labelled tickets.
What hardware is needed?
Training fits on a 12 GB GPU.
Where does training data come from?
An LLM writes or annotates it from your task description.
Related projects
Adapter that turns existing LLMs into Jev-style decision functions with option scores, optional calibration and review thresholds.
ProjectprototypePython- cbjev★ 4
Self-hosted encoder model giving Jev-compatible choice, score and noul answers in one pass, pitched as a better-calibrated Laya successor.
ProjectresearchPython Joke decision model trained to answer wrongly with total confidence: 1.9% accuracy at 0.957 mean confidence, via a Jev-style API.
ProjectresearchPythonPre-registered field test of Jev, GLiNER2.5-Decide, Laya and other decision models against small LLMs on fresh September 2026 data.
ProjectresearchPython11 MB CPU-only model that answers choice, noul and score questions with fitted calibration and no LLM at inference.
ProjectresearchPythonTiny pre-registered experiment testing whether a confidence-scoring reward (RLCD-like) yields confidences safe to automate on at 0.9.
ProjectresearchPython