jev-gateway (dragpk247)
Self-hosted LLM gateway with semantic cache, privacy sandbox and a 'Jev classifier' tiering requests across cheap and frontier models.
Overview
Jev Gateway (dragpk247) is a Docker-deployed OpenAI-compatible proxy aimed at cutting API costs. Requests pass a semantic cache, then a privacy sandbox that sends anything containing secrets or PII to a local GPU model, and otherwise a classifier labelled 'Jev Classifier' routes them to a cheap tier (such as DeepSeek-Chat or Gemini Flash) or a frontier tier for hard reasoning. It adds multi-provider failover on rate limits, a code gate that checks generated code syntax and retries a higher tier on failure, and a distillation harvester that collects traces to fine-tune local LoRA adapters.
The README makes marketing claims such as 70 to 85 percent cost reduction and 99.99 percent uptime, which are not backed by measurements in the excerpt. It is also not clear from the excerpt whether the classifier calls TypeSafe's Jev or a local component under the same name.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which tier should handle this request: fast cheap model or frontier model?(inferred) | — |
| Yes / no | Does this request contain secrets or PII?(inferred) | — |
How it works
- App sends a request to the gateway
- Semantic cache returns hits immediately
- Privacy check routes sensitive requests to a local GPU
- Classifier picks a tier; failover and code checks apply
Strengths
- Combines caching, privacy routing and tiering
- Local fallback for sensitive data
Caveats
- Cost and uptime claims unverified
- Unclear whether TypeSafe's Jev is used
- Many moving parts to operate
Getting started
`git clone https://github.com/dragpk247/jev-gateway.git && cd jev-gateway && docker compose up -d`.At a glance
- Type
- Service
- Status
- prototype
- Language
- Python
- Runtime
- Docker Compose proxy with local GPU
- Has code
- yes
Use cases and integrations
How curated lists describe it
Sub-300ms intelligent LLM router with Zero-Leak privacy sandbox, Obsidian RAG…
wh000wh000/awesome-jev-liveSub-300ms intelligent LLM router with Zero-Leak privacy sandbox, Obsidian RAG, and multi-tier local GPU & cloud offloading.
hellogumbo/awesome-jev
Frequently asked questions
What does the classifier decide?
Which model tier should serve each request.
Are the savings measured?
Not in the README excerpt; figures are claims.
Does it use TypeSafe's Jev?
The excerpt names a 'Jev Classifier' but does not show which model backs it.
Related projects
Claude Code plugin forked from jev-router that asks Jev how much effort each typed prompt needs and applies it to the turn.
ProjectbetaTypeScriptLocal-first routing engine for long-running agent tasks that uses a Jev decider to pick a model group per turn.
PluginproductionRust, TypeScriptOpen-source AI gateway for 100+ LLMs that can use Jev to classify requests for complexity routing and to compact tool results.
ProxyproductionPythonOpen-source workspace for running and reviewing AI coding agents across devices, with an optional Jev-based automatic model router.
ProjectproductionTypeScript- jev-router★ 593
Per-turn model routing for Claude Code and Codex: wraps the real CLI and lets Jev send simple turns to fast models and hard ones to strong.
ProxybetaJavaScript - jev-codex-router★ 274
Per-call router for Codex where Jev picks the model and thinking effort for every model call, including tool continuations.
ProjectbetaPython