laya-rs
From-scratch Rust reimplementation of Laya on candle, including the encoder, decision head, calibration, RLCD training and a server.
Overview
laya-rs (apiplant) is a from-scratch Rust reimplementation of Laya on Hugging Face's candle, with no PyTorch or Python at inference time. It targets the published Laya checkpoint family and rebuilds the full stack: a config-driven ModernBERT encoder (RoPE, alternating global and sliding-window attention, GeGLU), the decision head with option-marker scoring, a typed-question schema builder matching the original token layout and truncation rules, per-question-type and option-count temperature calibration with a fitting utility, and language routing using a Unicode script detector plus language identification so routing never relies on model confidence.
Unusually, it also implements an RLCD training loop: REINFORCE with Gaussian exploration and a group-mean baseline over the same strictly proper scoring rule reward as the original, with TD(lambda) for multi-turn episodes. It can run as an HTTP server with laya serve, downloading the typed-decisions checkpoint by default. A linked gist compares it with the Jev API and a GLiNER baseline on a typed-decisions fixture.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which option fits the state? | — |
| Score | Where does the state fall on the scale? | — |
| Yes / no | Is this statement true of the state? | — |
How it works
- laya serve downloads and loads a checkpoint
- Requests with state and typed questions are rendered to the original layout
- Encoder and decision head score options; calibration applies
- Answers return over HTTP
- Optionally train with the RLCD loop
Strengths
- No Python at inference
- Includes training as well as inference
- Explicit language routing
- Comparison with Jev and GLiNER published
Caveats
- Reimplementation may diverge from upstream over time
- Training loop is for advanced users
- Benchmark is a single fixture
Getting started
laya serve (binds 127.0.0.1:8000 and downloads the typed-decisions checkpoint on first use).At a glance
- Type
- Model
- Status
- beta
- Language
- Rust
- Runtime
- candle library, CLI and HTTP server
- Has code
- yes
Use cases and integrations
How curated lists describe it
Rust reimplementation of Laya, a sub-35ms non-autoregressive typed-decision…
wh000wh000/awesome-jev-live
Frequently asked questions
Is it a wrapper around Laya's Python code?
No, it reimplements the encoder, head, calibration and training in Rust.
Can it train models?
Yes, it includes an RLCD training loop.
How does it route languages?
With a Unicode script detector and language identification rather than model confidence.
Related projects
llama.cpp-based C++ server that turns any LLM into a System One decision model, reading answers from next-token logits.
ProjectbetaC++MLX library that turns an existing local model into a decision engine by reading candidate logits, with no fine-tuning, on Apple Silicon.
ProjectprototypePythonLocal server (xks) implementing Jev's /v1/systemone contract on an open-weight LLM, accelerated with Xeon Phi cards.
ProjectresearchRustEducational C11 library and CLI that turns DiffusionGemma 26B into a CPU-first OpenJev-style decision engine reading candidate logits.
ProjectresearchCExperimental engine that makes an existing LLM do Jev-like classification and scoring by reusing context and building JSON.
ProjectresearchPythonReads typed decisions from one forward pass of PrismML's 1.75-bit Bonsai 27B under llama.cpp, using the OpenJev/SemIf logit method.
ProjectresearchPython