jev-model-tokengate
OpenAI-compatible streaming proxy that holds tokens in a buffer and releases them only after a Jev check passes, cutting leaks pre-screen.
Overview
tokengate is an OpenAI-compatible proxy between an LLM and its users. Instead of redacting after text has been shown, it holds outgoing tokens in a small sliding buffer and scores each window against a criteria matrix in parallel while the response is still streaming. Only on all-pass is the batch released; on a violation the buffer is dropped, the client stream is cut and an abort is sent upstream so generation stops.
The README argues that conventional post-hoc guardrails leak 18-35 sensitive tokens before redaction, and that with buffering leakage becomes independent of evaluator latency: a slow engine makes the stream stutter but does not leak more. Its reported results are zero leaked characters across 5 scenarios and 3 engines (Jev noul and Claude with strict tool use among them), well-formed verdicts every time, and 5/5 classification, while the 35 ms latency target was missed at about 300 ms because of network round trip. Violations surface as a standard content_filter finish reason.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Does this window of tokens violate a criterion (for example, leak a secret)? | yes · no |
How it works
- Point the app's OpenAI base URL at the proxy
- Upstream tokens fill a sliding buffer
- Each window is scored against all criteria in parallel
- All-pass releases the batch verbatim; a violation cuts the stream and aborts upstream
Strengths
- Preventive rather than post-hoc moderation
- Leakage does not depend on evaluator latency
- One-line integration; upstream chunks replayed verbatim
- Stops paying for generation after a violation
Caveats
- Adds about 300 ms per window over the public internet in the author's tests
- Stream may stutter while windows are evaluated
- Small scenario set (5 scenarios)
Getting started
docker build -t tokengate . && docker run -p 8787:8787 -e JEV_API_KEY=... -e UPSTREAM_URL=https://api.openai.com/v1/chat/completions tokengateAt a glance
- Type
- Proxy
- Status
- prototype
- Runtime
- Docker proxy (OpenAI-compatible)
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
An OpenAI-compatible proxy that sits between your LLM and your users. It evaluates each sliding window of tokens \*\*while the response is still streaming\*\* and cuts the stream \*\*before\*\* a violating token can reach the screen.
logicrw/awesome-jev-projectsAn OpenAI-compatible proxy that sits between your LLM and your users. It evaluates each sliding window of tokens while the response is still streaming and cuts the stream before a violating token can reach the screen.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide?
Whether each buffered window of streamed tokens violates a criterion before it is released.
Why not moderate after display?
The README argues sensitive tokens have already been seen by then; holding them in a buffer keeps them off screen.
How fast is it?
The author measured about 300 ms per check, mostly network round trip, missing a 35 ms target.
Related projects
Claude Code plugin that checks every reply and file edit against your CLAUDE.md rules with one Jev yes/no per rule.
PluginbetaTypeScriptClaude Code plugin with nine Jev-judged gates for rules, scope, intent, completion and claims that escalate but never approve.
HookbetaTypeScriptSafety gate plugin for Claude Code, Codex CLI and Antigravity: local sensitive-file checks, then Jev judges blast radius.
PluginbetaTypeScriptClaude Code mod that answers auto mode's tool-call permission question with eight Jev yes/no checks behind a shell-aware blocklist.
ProjectbetaTypeScriptRing Zero: kernel-level runtime security for AI agents, with Jev as an optional check that can only raise severity, never permit.
ServicebetaClaude Code hook that blocks known key formats locally and sends unknown high-entropy strings to Jev only in masked form.
HookbetaPython