Parallel Constrained Decoding (Qwen2.5-1B-RLCD)
Hugging Face Space for parallel constrained decoding on Apple Silicon: fills multi-field JSON schemas in parallel with field confidence.
Overview
This Hugging Face Space (a Gradio app) demonstrates Parallel Constrained Decoding, a high-throughput engine on Apple Silicon using MLX for structured extraction, decision routing and categorical classification. Instead of generating JSON token by token, it prefills the context once into a shared KV cache, then evaluates each bounded field (booleans or enums) in parallel by slicing logits to the allowed values, giving schema-valid output by construction and per-field confidence.
The README reports, with Qwen2.5-1.5B-Instruct-4bit on an M4 Max, 5.6x to 7.0x lower latency than autoregressive decoding across fintech fraud routing, code security audit, a 255-choice tariff router and a 28-field support triage (for example 270 ms against 1,900 ms). The list summary calls it RLCD-trained, but a separate Java port notes that the associated Qwen-2.5-1B-RLCD repository contains no weights or fine-tune, only a custom decoding strategy over the stock model. It is useful as an alternative to Jev for bounded extraction.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Risk level for this transaction | — |
| Yes / no | Does this require review? | true · false |
| Choice | Which of 255 tariff codes applies? | — |
How it works
- Prefill the context once.
- Broadcast the KV cache to each schema field.
- Slice logits to each field's allowed values.
- Return schema-valid JSON with field-level confidence.
Strengths
- 5.6-7.0x faster than autoregressive JSON per the README.
- Schema validity by construction.
- Runs on Apple Silicon with a small model.
Caveats
- Despite the RLCD name, it uses a stock model with custom decoding.
- Apple Silicon (MLX) only.
Getting started
Open the Hugging Face Space at huggingface.co/spaces/drinkmoonshine/parallel-constrained-decoding.At a glance
- Type
- Model
- Status
- research
- Language
- Python
- Runtime
- MLX, Gradio
- Has code
- yes
- License
- Apache-2.0
Use cases and integrations
How curated lists describe it
Open research: RLCD-trained Qwen2.5-1B demo exploring open-source parallel constrained decoding as an alternative to Jev.
yibie/awesome-jevHugging Face Space exploring open-source parallel constrained decoding as an alternative to Jev.
hellogumbo/awesome-jev
Frequently asked questions
How is it faster than JSON mode?
It evaluates all bounded fields in parallel from one shared prefill instead of generating tokens sequentially.
Is the model fine-tuned?
Per a related port's notes, the underlying repo uses stock Qwen2.5-1.5B-Instruct with custom decoding.
What hardware does it need?
Apple Silicon with MLX.
Related projects
Java (llama.cpp via FFM) and Python engines that fill boolean/enum JSON schemas in two forward passes with per-field confidence.
ProjectresearchJavallama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++Python library that imitates Jev-style typed decisions (choice, score, yes/no) on local LLMs via Ollama, vLLM or llama.cpp.
ProjectprototypePythonMLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonTurns any local LLM into a Jev-style engine via masked-logit softmax, or proxies OpenAI-compatible APIs or Jev; includes a chat crush radar.
ServiceprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRust