vLLM Jev
Native vLLM serving for Jev-style decision models on Linux and Apple Silicon, including image and video questions for Valen.
Overview
vLLM Jev serves compatible Jev-style checkpoints through vLLM: give it a question and candidate answers and it returns a label and a probability for each candidate over HTTP, with Choice, Noul and Score supported. On Linux it uses vLLM's scheduling, batching, compilation, KV cache and metrics; an Apple Silicon preview runs supported text models via MLX or PyTorch MPS and Valen multimodal decisions via MLX.
Supported multimodal models include Valen, vjev, RSI-Jev v4.0-VL, JevAny Pointer and Clef. The README demos include Valen answering 24 questions about one video in 1.22 s versus 5.31 s on its PyTorch reference path, with all answers agreeing; local image decisions on an M5 Mac; Sokoban play; and 40 concurrent text Choice requests compared with a reference server.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which candidate answer is correct for this question (text, image or video)? | — |
| Yes / no | Yes/no question | yes · no |
| Score | Ordered level question | — |
How it works
- Install vLLM Jev and load a supported checkpoint.
- Send questions and candidates over HTTP, optionally with images or video.
- vLLM batches and scores candidates.
- Return labels and per-candidate probabilities.
Strengths
- Reported 4.3 times speedup over a reference path on video questions.
- Production serving features from vLLM.
- Apple Silicon preview.
Caveats
- Only compatible checkpoints are supported.
- Apple Silicon support is a preview.
At a glance
- Type
- Service
- Status
- beta
- Language
- Python
- Runtime
- vLLM server, MLX/MPS on Apple Silicon
- Has code
- yes
- Stars
- 524
Use cases and integrations
How curated lists describe it
Native vLLM serving for decision models on Linux and Apple Silicon, including image and video questions for Valen
AnotiaWang/awesome-decision-modelsNative vLLM serving for Jev decision models
wh000wh000/awesome-jev-liveNative vLLM serving for Jev decision models.
hellogumbo/awesome-jevServes decision models natively on vLLM, answering candidate probabilities over the System One endpoint.
MrJev/awesome-jev
Frequently asked questions
Which models are supported?
Jev-style checkpoints including Valen, vjev, RSI-Jev v4.0-VL, JevAny Pointer, Clef and Open-Jev-2B.
Does it run on a Mac?
Yes, as a preview via MLX or PyTorch MPS.
How much faster is it?
The README shows 1.22 s versus 5.31 s for a 24-question video example.
Related projects
Local Jev-compatible API on a Mac: Qwen via llama.cpp/Metal reads one output token per question for text and image decisions.
ModelbetaPythonHugging Face blog on llama.cpp's /v1/systemone endpoint serving decision models locally in Jev's wire format.
ServiceproductionC++- Lichen★ 58
Drop-in local replacement for Jev that serves /v1/systemone from open weights, including questions about images.
ServicebetaPython Local /v1/systemone server in Japanese docs with LFM, Sarashina and fine-tuned ModernBERT-Ja backends, including image input.
ServicebetaPythonHugging Face card for NeoHorse-Jev-4B: Apache-2.0 4B decision model for text or image-plus-text, served at /v1/systemone.
ModelbetaPythonStandard Thinking's Apache-2.0 3B and 8B decision models on Ministral 3 that score bounded answers via /v1/systemone, with vision.
ServicebetaPython