jev-smart-router
Databricks App that uses a Jev Choice over named routing options to pick which Foundation Model endpoint answers each message.
Overview
JEV Smart Router is a FastAPI-based Databricks App for model routing. You define named routing options such as Coding, General, or Cheap and fast, each with a plain-English description and a target model. For each chat message, the app posts the conversation and option descriptions to Jev as a choice question; Jev returns the winning label, a confidence and a probability distribution over options.
The winning option maps to a Databricks Foundation Model API serving endpoint, which the app invokes with its own service principal, so no per-user token is needed. The UI includes a routing-option manager and chat, and shows the routed model, Jev's confidence, a per-option probability tooltip, routing latency and model inference time. It is aimed at Databricks users who want an explainable, model-agnostic router rather than hard-coded rules.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which routing option's description best fits this conversation? | — |
How it works
- User sends a chat message
- Conversation plus {option_label: description} criteria posted to Jev as a Choice
- Winning option mapped to a Databricks FMAPI endpoint
- Inference runs via the app service principal; UI shows model, confidence and timings
Strengths
- Explainable routing with a full probability breakdown per option
- Routing options editable in plain English without code changes
- No per-user token needed for inference
Caveats
- Tied to Databricks Apps and FMAPI endpoints
- Routing quality depends on how distinct the option descriptions are
- No evaluation results reported
At a glance
- Type
- Service
- Status
- prototype
- Language
- Python
- Runtime
- Databricks App (FastAPI)
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
JEV Smart Router — a Databricks App that uses TypeSafe JEV to pick which model answers each message, then runs inference on the chosen Databricks Foundation Model API endpoint.
logicrw/awesome-jev-projectsJEV Smart Router — a Databricks App that uses TypeSafe JEV to pick which model answers each message, then runs inference on the chosen Databricks Foundation Model API endpoint.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide?
Which of your named routing options best fits the current conversation, with probabilities for each.
Who runs the actual model call?
Databricks: the chosen option maps to a Foundation Model API endpoint called with the app's service principal.
Can I change the routes?
Yes, the UI lets you manage routing options, each with a description and target model.
Related projects
Alibaba's AgentScope 2.0 agent framework, which added a JevClassifierModel for binary, choice and score decisions via the TypeSafe SDK.
ProjectproductionPythonLocal-first routing engine for long-running agent tasks that uses a Jev decider to pick a model group per turn.
PluginproductionRust, TypeScriptKilo Code's cloud monorepo, where a System One classification replaced an LLM call in the auto-routing path.
ProjectproductionTypeScriptOpen-source AI gateway for 100+ LLMs that can use Jev to classify requests for complexity routing and to compact tool results.
ProxyproductionPythonOpen-source workspace for running and reviewing AI coding agents across devices, with an optional Jev-based automatic model router.
ProjectproductionTypeScriptSelf-hosted Stacks blockchain data service whose Slack gate and fault-triage paths reportedly run on Jev decisions.
ServiceproductionTypeScript