toolJev
Code Mode for MCP where retrieval shortlists tools and Jev picks one or says none fits, benchmarked on several tool-use suites.
Overview
toolJev sits in front of all of an agent's MCP servers and replaces hundreds of tool schemas with two tools: search(query), where retrieval shortlists candidate tools and Jev decides whether any of them fits, and execute, which lets the agent run code that can call Jev per item instead of making one LLM call per item. The sub-model in this Code Mode setup is a decision model rather than an LLM.
The author benchmarked it on MCPToolBench++, LiveMCPBench, When2Call and live Claude Haiku agents, and reports where the design worked and where it did not. Reported results include 98.8 percent accuracy on 400 support tickets at 5.3 times lower cost than Haiku, 77 percent fewer tokens with 612 tools, and a finding that below about 100 tools direct tool calling was better. Sample sizes are small and documented in bench/RESULTS.md. A local stand-in, nanojev, is much less accurate.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which of the shortlisted tools fits this request? | — |
| Yes / no | Does any shortlisted tool fit this request? | yes · no |
| Choice | Per-item decisions such as routing a support ticket | — |
How it works
- The agent calls search(query).
- Retrieval shortlists tools and Jev picks one or reports that none fits.
- The agent calls execute with code that may call Jev per item.
- Confident answers (e.g. at least 0.9) are acted on in code; others go back to the agent.
Strengths
- The README reports 77 percent fewer input tokens with 612 tools.
- Reported 5.3 times cheaper ticket routing at similar accuracy.
- Jev's no-fit signal reached AUROC 0.94 on When2Call near-misses, per the README.
- Methods and caveats are documented in detail.
Caveats
- Agent runs use small samples (1 to 2 reps, 36 tasks).
- Slower than direct tool search in agent runs (34 s vs 18 s).
- Not worth it below about 100 tools, per the author.
Getting started
uv run python examples/showcase.py (local models, no API key)At a glance
- Type
- Benchmark
- Status
- research
- Language
- Python
- Runtime
- MCP server proxy
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
Code Mode for MCP, where the sub-model is a calibrated decision model (Jev), not an LLM. Benchmarked on MCPToolBench++, LiveMCPBench, When2Call and live Claude agents.
logicrw/awesome-jev-projects
Frequently asked questions
What does Jev decide in toolJev?
Which shortlisted tool fits a request, whether any fits, and per-item decisions inside executed code.
When is it worth using?
The author found it pays off at large tool counts; below about 100 tools direct calling did better.
Can I try it without a key?
Yes, the showcase example runs with local models through nanojev, which is less accurate.
Related projects
MCP server exposing Jev Noul, Choice and Score via OpenRouter so coding agents can make quick decisions without reasoning tokens.
MCP serverbetaTypeScriptMCP relay that adds use_jev to every server: Jev picks tool calls, the calling model writes free text, the relay executes.
MCP serverbetaTypeScriptMCP server exposing Jev as one evaluate tool, plus a harness comparing Jev's judgments with general LLMs on shared datasets.
MCP serverbetaJavaScriptLocal Python MCP service that keeps the skill catalog out of the agent's context and lets Jev pick which SKILL.md files to load.
MCP serverbetaPythonLocal gateway, MCP server and Claude Code hooks that let coding agents delegate explicit choices, classification and scoring to Jev.
MCP serverbetaTypeScriptExperiments testing Jev ideas against baselines, starting with Jev as a top-10 tool router for MCP agents on three benchmarks.
ProjectresearchPython