Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

jev-model-router

Web chat where one Jev Choice picks nano, fast, balanced or frontier model tier per message via the Vercel AI Gateway.

Serviceprototype
View on GitHub ↗Live demo ↗

Overview

Jev model router is a small web chat in which TypeSafe's Jev picks which LLM should answer each message, and the chosen model then replies. Both calls go through the Vercel AI Gateway with one key. It was inspired by Riley Brown's video on Jev and built step by step with Claude Code, with extensive write-ups of the development journey, a reliability upgrade and 18 'unknown unknowns'.

The server sends one Choice question with four tiers, each with a one-line rubric: nano (gpt-5-nano), fast (gemini-3-flash), balanced (claude-sonnet-5) and frontier. Jev returns a probability per tier and a confidence; the server calls the chosen model and returns the reply, probabilities and timings. The README estimates one routing decision at about 340 input tokens, roughly $0.000014. Offline tests cover held-out routing cases, and a live eval writes per-case results.

What the decision model decides

TypeQuestionAnswers
ChoiceWhich tier should answer this message?nano · fast · balanced · frontier

How it works

  1. The browser posts a message to /chat
  2. The server asks Jev one Choice over four tiers with rubrics
  3. Jev returns probabilities and confidence
  4. The server calls the chosen model and returns reply, probabilities and timings

Strengths

  • Very low routing cost per message
  • Tiers and question editable at the top of server.mjs
  • Offline tests and live eval without calling reply models
  • Detailed documentation of pitfalls

Caveats

  • Needs a paid Vercel AI Gateway key
  • Demo-scale chat, not a production router

Getting started

npm install, put AI_GATEWAY_API_KEY in .env, then node --env-file=.env server.mjs and open http://localhost:3000

At a glance

Type
Service
Status
prototype
Language
JavaScript
Runtime
Node web server
Has code
yes
Has tests
yes

Use cases and integrations

How curated lists describe it

Frequently asked questions

What does Jev decide?

Which of four model tiers should answer each chat message.

How much does routing cost?

The README estimates about $0.000014 per decision.

Why does it need a paid gateway tier?

The free tier limits Jev to a few calls and refuses most reply models.

Related projects

  • HiRoute

    Local-first routing engine for long-running agent tasks that uses a Jev decider to pick a model group per turn.

    PluginproductionRust, TypeScript
  • LiteLLM

    Open-source AI gateway for 100+ LLMs that can use Jev to classify requests for complexity routing and to compact tool results.

    ProxyproductionPython
  • openchamber

    Open-source workspace for running and reviewing AI coding agents across devices, with an optional Jev-based automatic model router.

    ProjectproductionTypeScript
  • jev-router★ 593

    Per-turn model routing for Claude Code and Codex: wraps the real CLI and lets Jev send simple turns to fast models and hard ones to strong.

    ProxybetaJavaScript
  • jev-codex-router★ 274

    Per-call router for Codex where Jev picks the model and thinking effort for every model call, including tool continuations.

    ProjectbetaPython
  • ailerix

    Model router where Jev classifies each request's task family and code picks the cheapest capable model on a cost Pareto chain.

    ProjectbetaTypeScript