第三方独立实测：Classmethod / DevelopersIO
Classmethod DevelopersIO independent test of Jev for LLM model routing, reporting 40 successful calls and 10 of 10 correct classifications.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
An article on Classmethod's DevelopersIO blog testing Jev for routing requests between LLMs. It is listed as an independent third-party trial.
Per the listing, all 40 calls succeeded and 10 of 10 classifications were correct in the author's test.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which model should handle this request?(inferred) | — |
How it works
- Prepare routing test cases
- Ask Jev to classify each
- Check success and correctness
Strengths
- Independent hands-on test
Caveats
- Small sample (10 classifications)
- Only list summary available
At a glance
- Type
- Article
- Status
- research
Use cases and integrations
How curated lists describe it
All 40 calls succeeded, with 10/10 classifications correct;
wh000wh000/awesome-jev-live
Frequently asked questions
What was tested?
Jev as a router choosing between LLMs.
What were the results?
40 of 40 calls succeeded and 10 of 10 classifications were correct, per the listing.
Where is it?
https://dev.classmethod.jp/en/articles/jev-for-llm-model-routing
Related projects
James Cruce's write-up on choosing local LAYA over hosted Jev: the 421M open model beat a home-made router on a frozen replay.
ArticlediscussionDeterministic evidence selector for noisy Codex investigations; its earlier Jev subagent router is kept for the record, not recommended.
ProjectresearchResearch framework (DART) testing whether Jev's probabilities are a reliable signal to answer, escalate or abstain.
ProjectresearchPythonBrowser playground: configure a pool of OpenRouter models, let Jev pick one for a task, then judge whether its pick was right.
AppprototypeJavaScriptPython tool that measures where to route decisions between Jev and a larger model on your own data, then routes by confidence.
BenchmarkbetaPythonJev as an LLM router on RouterArena and LLMRouterBench: the router beat the best single model, but a no-Jev ablation matched it.
BenchmarkresearchPython