jev-oncall
On-call tooling where Jev judges alerts and incident hypotheses and plain code decides: alert routing plus a no-LLM root-cause benchmark.
Overview
jev-oncall applies one rule throughout: the model judges, plain code decides. Its alert-triage server asks Jev one call of four typed questions per alert (actionable, severity, owning team, duplicate-of). Per the list summary, it pages when P(SEV1)+P(SEV2) is at least 0.80, drops only below 0.20 when actionability agrees, sends the middle band to a human with 15 minutes to acknowledge before paging anyway, links duplicates into a cycle-broken incident graph, and falls back to configured severity on errors or timeouts. A 300-alert run measured p50 418 ms, p95 1477 ms and $0.04 per 1,000 alerts.
The current focus is a root-cause analysis benchmark on 60 recorded faults from RCAEval microservice systems. A jev-agent mode with no general-purpose LLM, where Jev answers bounded questions and code picks checks and verifies, named the right service in 57 of 60 cases, matching LLM investigators while being 6-22 times faster at about $0.004 per case, though it cited the right kind of evidence less often.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this alert actionable? | yes · no |
| Score | What severity is this alert?(inferred) | SEV1 · SEV2 · SEV3 · SEV4 |
| Choice | Which team owns this alert? | — |
| Choice | Which hypothesis does the evidence support / which check next? | — |
How it works
- Send each alert to Jev with four typed questions
- Page, drop or send to a human review band based on probabilities
- Link duplicates into an incident graph without cycles
- Fall back to configured severity on error
- For RCA, let Jev pick checks and hypotheses while code enforces budgets and verification
Strengths
- Fail-open defaults never silence alerts on errors
- Human review band with an escalation timer
- Published latency, cost and RCA traces
- Verified RCA answers were all correct in the reported run
Caveats
- RCA cites weaker evidence than LLM investigators
- Benchmark limited to two microservice systems
- Requires a TypeSafe key
At a glance
- Type
- Project
- Status
- beta
- Language
- Python
- Runtime
- Server and benchmark harness
- Has code
- yes
- Has tests
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
On-call operations: asks Jev one call of four typed questions per alert (`Noul` actionable, `Score` severity, `Choice` owning team, `Choice` duplicate-of), pages on P(SEV1)+P(SEV2) ≥ 0.80 and drops only below 0.20 when actionability agrees, sends the band between to a human who has 15 minutes to ack before it pages anyway, links duplicates into a cycle-broken incident graph so two alerts can never silence each other, and falls back to configured severity on any error or timeout; a published 300-alert run measured p50 418 ms, p95 1477 ms and $0.04 per 1,000 alerts, with the slowest call landing 151 ms short of the 2 s timeout.
yibie/awesome-jevIncident triage on TypeSafe Jev — the model judges, plain code decides. Routing on probability distributions with a human-review middle band and fail-open defaults.
logicrw/awesome-jev-projects
Frequently asked questions
How are alerts routed?
By probability: page above 0.80 combined SEV1+SEV2, drop below 0.20 with agreeing actionability, and send the rest to a human.
Can Jev find root causes without an LLM?
In the reported benchmark it named the right service in 57 of 60 cases, though with weaker cited evidence.
What happens if Jev fails?
The system falls back to the alert's configured severity.
Related projects
grep by meaning: pipe text in, ask a yes/no question in plain English, and Jev returns matching lines, even behind tail -f.
BenchmarkbetaPythonSREGym blog post on Kubernetes incident diagnosis with no LLM agent: Jev picks a root cause from collected evidence.
ArticleresearchFifteen live arcade games where Jev and LLMs play each other, led by WikiRace in which Jev picks only real links.
ProjectbetaTypeScriptMilvus-based multi-hop graph RAG library where two-stage Jev Noul scoring selects relations and reranks passages.
LibrarybetaPythonExperiment testing whether a local Qwen read via label log-probabilities can replace Jev on synthetic AIOps alert decisions.
ProjectresearchPythonWeb app that sends identical ticket-routing and alert-triage questions to OpenAI Decisions and Jev side by side.
BenchmarkprototypePython