openjev-sglang
Archived early Jev-compatible API server running Qwen3.6-35B-A3B prefill-only on SGLang via Modal; superseded by SGLang's native endpoint.
Overview
openjev-sglang was an early experiment to reproduce Jev after its launch: a server implementing the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang, using prefill-only inference. Each container runs on one B200 with SGLang 0.5.19's Rust frontend, radix caching and breakable prefill CUDA graphs, with a separate FastAPI process handling the API.
It deploys on Modal as an autoscaling server that scales to zero after five idle minutes, persists weights and compilation caches in a Modal volume, exits cleanly if SGLang dies, and works around an SGLang mixed-logprob batch crash during warmups. A smoke command tests a deployed endpoint. The README notes the project is now superseded because SGLang ships a native decisions endpoint implemented the same way, and recommends using that instead.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Other | Choice, Score and Noul questions in the Jev HTTP API format | — |
How it works
- uv sync and authenticate Modal.
- uv run modal deploy modal_app.py to deploy.
- Send Jev-format requests to the public endpoint.
- Run uv run openjev smoke against it.
Strengths
- Showed prefill-only Jev compatibility on SGLang.
- Production-minded Modal deployment.
Caveats
- Archived and superseded by SGLang's native decisions endpoint.
- Requires B200 GPUs on Modal.
Getting started
Use SGLang's native decision-models endpoint instead (docs.sglang.io); to try this: uv sync && uv run modal deploy modal_app.py.At a glance
- Type
- Service
- Status
- research
- Language
- Python
- Runtime
- SGLang, FastAPI, Modal
- Has code
- yes
- Has tests
- yes
- Stars
- 333
Use cases and integrations
How curated lists describe it
Jev-compatible API endpoint based on open models (prefill-only)
wh000wh000/awesome-jev-live*.
wh000wh000/awesome-jev-liveJev-compatible API endpoint backed by open models and prefill-only inference.
cobanov/awesome-jev*(archived)* - Jev-compatible API server running an open model on SGLang.
MrJev/awesome-jev
Frequently asked questions
Is it still maintained?
No, it is archived; SGLang now has a native decisions endpoint.
Which model did it use?
Qwen3.6-35B-A3B on SGLang.
Where does it run?
On Modal with B200 GPUs.
Related projects
Docker service exposing local Laya, Von, a CLM head or hosted Jev through one TypeSafe-compatible HTTP and MCP endpoint.
MCP serverbetaPythonConverts open Jev reproductions (Laya, Kev, NanoJev, PlayJev) to INT8 ONNX for offline CPU decisions at about 15.6 ms per question.
ServicebetaPythonNative Go runtime serving Gemma 4 12B as a Jev-like decision model with hand-tuned PTX kernels and no CGo or CUDA toolkit.
ProjectbetaGoGo harness for System One models: hosted Jev, open Kev in-process via llama.cpp, and Cloudflare Clef, with typed helpers and a CLI.
ServicebetaGoGo gateway exposing the Jev /v1/systemone contract over ordinary LLM backends by reading single-token label logprobs.
ServicebetaGoRuns Intern-Decision-0.8B on Apple Silicon behind a local /v1/systemone endpoint that accepts screenshots for agent decisions.
ServicebetaPython