laya-rs
CPU-first Rust runtime and HTTP server for Laya's multilingual model on ONNX Runtime, with portable builds and Chinese and English docs.
Overview
This laya-rs (redwolf2019) is a Rust inference runtime and HTTP server, laya-server, for self-hosting Laya on Linux, Windows or macOS CPUs. You submit a state and typed questions and get back a choice with per-option probabilities, a score as an expected level with distribution, or a probability for a yes/no proposition. It defaults to the multilingual ONNX bundle, so it handles Chinese, English and other supported languages, with the model resident in memory and shared by HTTP clients. It adds shared bearer-token auth, health checks, metrics, bounded queues and graceful shutdown.
The documentation, primarily Chinese, is careful about compatibility: it is not a drop-in Jev replacement. Its endpoint is /v1/system-one with a hyphen, it loads one fixed model with no per-request model selection, it requires instructions for every question, it keeps Laya's act_probability extension field, and its compatibility baseline is a pinned Laya commit. It has not been compared with Jev on answers or performance. Release 0.1.4 ships four native packages, with validation scope and failures documented.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which department should handle this ticket? | — |
| Score | How high is the processing priority? | — |
| Yes / no | Does this need human intervention? | — |
How it works
- Install the native package for your platform and download the model
- Start laya-server with a bearer token
- POST state and questions to /v1/system-one
- Use typed answers; monitor via health and metrics endpoints
Strengths
- Runs on CPU across Linux, Windows and macOS
- Operational features such as auth, metrics and bounded queues
- Honest documentation of compatibility gaps and validation
Caveats
- Not a drop-in Jev replacement
- Fixed single model
- Some platform validations failed and are documented
Getting started
Download release v0.1.4 for your platform and follow the installation guide in docs/installation.md.At a glance
- Type
- Service
- Status
- beta
- Language
- Rust
- Runtime
- HTTP server on ONNX Runtime (CPU)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Pure-Rust runtime and HTTP server for Laya on ONNX Runtime, CPU-first, defaulting to the multilingual model, with portable builds for Linux, Windows and macOS. Chinese and English docs.
MrJev/awesome-jev
Frequently asked questions
Can I point my Jev client at it?
Not directly; the endpoint path, model handling and validation differ, as the README documents.
Does it need a GPU?
No, it is CPU-first.
Which model does it run?
A fixed laya-multilingual ONNX bundle.
Related projects
Docker service exposing local Laya, Von, a CLM head or hosted Jev through one TypeSafe-compatible HTTP and MCP endpoint.
MCP serverbetaPythonConverts open Jev reproductions (Laya, Kev, NanoJev, PlayJev) to INT8 ONNX for offline CPU decisions at about 15.6 ms per question.
ServicebetaPythonNative Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo harness for System One models: hosted Jev, open Kev in-process via llama.cpp, and Cloudflare Clef, with typed helpers and a CLI.
ServicebetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoRuns Intern-Decision-0.8B on Apple Silicon behind a local /v1/systemone endpoint that accepts screenshots for agent decisions.
ServicebetaPython