MorrowCache
OpenAI-compatible chat cache proxy that asks Jev or a local System One model whether a new question has the same intent as a cached one.
Overview
MorrowCache (npm package @kushalicious/jevcache) is a local OpenAI-compatible chat response cache. Instead of cosine similarity, it asks a decision model whether a new request has the same intent as one already answered, and replays the stored answer when it does. Version 2.0 also catches agents retrying the same job with the same prior conversation.
The default adjudicator is TypeSafe Jev via OpenRouter; it can also point to local Kev, Laya, Laya MLX or any System One-compatible server, which removes the need for an OpenRouter key for admits. It stores only final assistant text, never responses with tool_calls, fails open if the adjudicator errors, and refuses stale answers based on freshness rules. The README reports a real-app example of a 3244 ms miss becoming a 394 ms hit at intent 0.93. It uses Node's built-in sqlite.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this request the same intent as a cached one? | yes · no |
How it works
- Client sends a /v1/chat/completions request to the proxy.
- Adjudicator decides whether it matches a cached question's intent.
- On a confident match within freshness limits, replay the cached answer.
- Otherwise call the upstream model and store the final text.
Strengths
- Large latency and cost savings on repeated questions (README example: about 3.2 s to 0.4 s).
- Fails open, so adjudicator errors never block a call.
- Can run fully locally with Kev or Laya.
Caveats
- A wrong same-intent judgment returns a cached answer to a different question.
- Upstream chat still needs a real model key.
Getting started
npx @kushalicious/jevcache@latest start (set ADJUDICATOR=kev or laya for a local judge).At a glance
- Type
- Proxy
- Status
- beta
- Language
- TypeScript
- Runtime
- Node proxy, Docker
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
MorrowCache — skip the chat call when the question is the same. OpenAI-compatible proxy. npm: @kushalicious/jevcache
logicrw/awesome-jev-projectsSkip expensive LLM calls when TypeSafe Jev says same intent. OpenAI-compatible local cache proxy — npx @kushalicious/jevcache.
hellogumbo/awesome-jevOpenAI-compatible proxy that asks a System One model whether a new question is the same one it has already answered, and replays the cached reply when it is. Not the same project as the jevcache above.
MrJev/awesome-jev
Frequently asked questions
How is it different from a semantic cache?
It asks a calibrated decision model whether intents match rather than relying on embedding similarity.
Do I need OpenRouter?
Only for the default Jev adjudicator; local Kev, Laya or other System One servers do not need it.
What if the adjudicator is down?
The proxy fails open and calls the upstream model.
Related projects
Local Jev-compatible proxy that records an app's typed decisions, distils a head per decision site on Laya, and answers when confident.
ProxybetaPython- Jevstiller★ 68
Proxy that distils a repeated Jev classification task into a local model on the fly and answers most requests itself once it agrees.
ModelbetaPython Single-binary local cache that memoizes Jev-class decisions on model, schema and redacted state, for cheap repeats and CI replay.
ModelbetaRustPre-flight hook for WorkBuddy and Hermes Agent where Jev classifies each prompt on nine dimensions and injects routing cues.
ProjectbetaPythonTiny JavaScript semantic matcher that classifies text locally in microseconds and escalates ambiguous cases to Jev.
ProjectbetaJavaScriptTransparent proxy that asks Jev to classify each coding-agent prompt and injects the decision into the agent's context before it generates.
ProxybetaPython