Laya-CoreML
Core ML and Neural Engine port of Laya for Apple Silicon, about 5 ms per short decision on an M3 Max with energy benchmarks.
Overview
Laya-CoreML runs open-weight Laya typed decisions on Apple Silicon through Core ML and the Neural Engine, with zero generated tokens and no PyTorch, Transformers or MLX needed for inference. Version 0.4.0 aligns directly with official Laya 0.4.1, sharing its batch and long-text prediction, hooks, structured decisions, calibration, HTTP and MCP servers, evaluation and router behaviour, with a migration guide and documented Core ML boundaries.
The README reports one short multilingual decision at 4.98 ms median and 5.31 ms p95 on an M3 Max with Neural Engine FP16, and 2.78 times better whole-system energy per decision than compiled MLX FP16, while noting that a requested tenfold improvement was not achieved. A Snake demo plays with visible probabilities and a safety layer, sustaining about 49 to 50 decisions per second across three 600-step episodes with zero deaths and two safety interventions. It comes from the author of laya-mlx and is on PyPI with weights on Hugging Face.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which direction should the snake move next?(inferred) | — |
| Yes / no | Does the customer request a refund of a duplicate payment? (example)(inferred) | — |
How it works
- pip install laya-coreml and download Core ML weights
- Load the model with laya_coreml.load
- Call predict with a state and typed questions
- Model runs on the Neural Engine and returns probabilities
Strengths
- Very low latency on Apple Silicon
- Energy efficiency measured against MLX
- No PyTorch at inference time
- Aligned with official Laya semantics
Caveats
- Apple Silicon and macOS 15+ only
- First Core ML initialisation can take tens of seconds
- Some Core ML boundaries remain versus upstream
Getting started
pip install laya-coreml, then laya.load("aac6fef/laya-multilingual-coreml-ane") and agent.predict(state, questions).At a glance
- Type
- Benchmark
- Status
- beta
- Language
- Python
- Runtime
- Core ML on Apple Silicon (macOS 15+)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Core ML and Neural Engine port of Laya from the Laya-MLX author, with validated ports and reproducible speed and energy benchmarks; about 5 ms per short decision on an M3 Max
AnotiaWang/awesome-decision-models
Frequently asked questions
How fast is it?
About 5 ms per short multilingual decision on an M3 Max, per the README.
Does it need PyTorch?
No, inference uses Core ML only.
Is it compatible with official Laya?
Version 0.4.0 aligns with Laya 0.4.1 semantics, with some documented Core ML limits.
Related projects
Docker recipes and ARM64 images for serving ten native decision checkpoints, such as Cloudflare Clef, behind /v1/systemone on DGX Spark.
ServicebetaPythonCUDA-native engine that serves open Qwen3.5 checkpoints behind a Jev-compatible API, reading option logits without decoding.
BenchmarkbetaC/CUDAToy local Jev-shaped decision API on a 0.5B model that answers yes/no, choice and score questions by scoring option tokens.
ServiceprototypePythonSmall open decision models (0.8B and 4B, from Qwen3.5) with a server that speaks Jev's /v1/systemone format in one forward pass.
ServicebetaPythonServes MiniCPM5-2B-8bit on Apple Silicon as a Jev-compatible /v1/systemone service and reproduces its JevBench score.
ModelprototypePython- reflex★ 166
Small open decision model on a frozen Qwen3.5-4B serving /v1/systemone in about 200 ms, benchmarked against Jev on JevBench.
ServiceresearchPython