GPTCache
Zilliz semantic cache for LLM queries, listed for a Jev evaluator that uses Noul checks to decide whether a cached answer can be served.
Overview
GPTCache is Zilliz's library for building a semantic cache for LLM queries, aimed at cutting API costs and latency by serving similar requests from cache. It integrates with LangChain, offers a server Docker image for use from any language, and recommends its generic get and set API since it no longer adds adapters for new model APIs.
According to the list summaries, it includes a Jev evaluator that uses Noul judgments to check whether a cached response can serve an incoming request, including cache-hit freshness and time-dependent query validity. This addresses a known weakness of similarity-only caching, where a semantically similar but outdated or time-sensitive answer is returned. The README reviewed does not describe the Jev evaluator. It installs with pip install gptcache.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Can this cached response correctly serve the incoming request? | yes · no |
| Yes / no | Is this query time-dependent such that the cached answer may be stale?(inferred) | yes · no |
How it works
- Incoming query finds a similar cached entry
- Jev evaluator judges whether the cached answer still applies
- Serve from cache or call the LLM
- Store new answers in the cache
Strengths
- Reduces stale or mismatched cache hits
- Keeps cost savings of semantic caching
Caveats
- Jev evaluator not described in the README
- Project no longer adds new model adapters
Getting started
pip install gptcache and configure the Jev evaluator per the project documentation.At a glance
- Type
- Library
- Status
- production
- Language
- Python
- Runtime
- library / server
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Semantic caching: Zilliz semantic cache integrates TypeSafe Jev Noul checks to evaluate cache hit freshness and time-dependent query validity.
yibie/awesome-jevSemantic cache with a Jev evaluator that uses Noul judgments to check whether a cached response can serve an incoming request.
logicrw/awesome-jev-projects
Frequently asked questions
What does Jev check?
Whether a cached response can validly serve a new request, including freshness.
Why not similarity alone?
Similar queries can still need different or more recent answers.
How do I install GPTCache?
pip install gptcache.
Related projects
Local MCP bridge letting Codex hand drafting to Gemini via a Google AI Pro quota, with Jev checking drafts against explicit criteria.
MCP serverbetaCost-, cache- and quota-aware LLM router that classifies each turn with a Jev-class model, prices every route, and checks cheap answers.
ProjectprototypePythonOpenRouter cookbook: Jev checks a cheap model's draft against grounding constraints before accepting it or escalating to a stronger model.
ArticleHack Club's lightweight LLM proxy, listed for adding a Jev forwarding endpoint behind its auth, limits and usage logging.
MCP serverproductionTypeScriptnpm library that rate-limits, spend-caps, caches and redacts every Jev request made in a loop.
LibrarybetaJavaScriptDeepSeek Harness Web GUI plugin giving the agent a jev_ask tool, usage tracking and an optional command guard with explicit caveats.
PluginbetaTypeScript