jev-qa-demos
TypeScript demos for testers: Jev triages CI test failures, gates test-agent commands and RAG answers, across gateway, Ollama and Foundry.
Overview
jev-qa-demos is the code for a video series explaining Jev to QA and SDET engineers, written as simple, commented TypeScript using the AI SDK's experimental_evaluate through Vercel AI Gateway. The main demo triages eight failed CI tests: Jev picks the root cause (Choice), rates release impact (Score) and estimates whether a retry will pass (Noul), and code routes each to file a bug, fix the test, ping DevOps, auto-retry or human review.
Other demos ask whether a test agent should run a command, check whether a RAG answer is grounded as an LLM-as-judge replacement, probe Jev's documented weak spots (maths, dates, counting, double negation, prompt injection) and show fixes, run the same triage locally through Ollama's systemone endpoint, run it on Microsoft Foundry's Decision-1 model, and compare it with an LLM on latency, tokens and cost. Table-driven unit tests cover the routing rules with a mock model and no API calls.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | What is the root cause of this test failure? | — |
| Score | How much does this failure impact the release? | — |
| Yes / no | Will a retry pass? | yes · no |
| Yes / no | Is this RAG answer grounded in the sources? | yes · no |
How it works
- Load eight failed CI tests with known labels
- Ask Jev root cause, release impact and retry questions
- Route each failure in code
- Compare with local Ollama, Foundry and an LLM
Strengths
- Clear, test-oriented examples
- Runs on hosted, local and Foundry decision models
- Unit tests without API calls
- Demonstrates known weak spots
Caveats
- Demo code for a video, not a library
- Vercel gateway needs a card on file
- Free promotion referenced in the README has ended
Getting started
npm install, cp .env.example .env (add key), npm run demo:helloAt a glance
- Type
- Library
- Status
- prototype
- Language
- TypeScript
- Runtime
- Node scripts (AI SDK)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Demos for the video "Jev explained for testers": TypeSafe AI's System One model, called through \*\*Vercel AI Gateway\*\* with the AI SDK's \`experimental\_evaluate\`.
logicrw/awesome-jev-projectsJev (TypeSafe AI) demos for QA / SDET engineers via Vercel AI Gateway - simple, commented TypeScript for a video walkthrough.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide in the triage demo?
Root cause, release impact and whether a retry will pass for each failed test.
Can I run it locally?
Yes. One demo uses Ollama's /v1/systemone endpoint with no API key.
Does it compare with an LLM?
Yes, on latency, tokens, cost and parsed versus native output.
Related projects
Small Python demos of Jev: parallel Noul ticket checks with measured speedup, and a three-way Choice citation and claim checker.
AppprototypePythonEducational code where a local LLM voices a game NPC and Jev classifies player messages and checks every line for spoilers.
ProjectprototypePythonOfficial TypeSafe cookbook example for checking whether source context actually supports a claim.
ArticleproductionSemantic test matchers for Vitest and Jest that ask Jev whether output satisfies a plain-English expectation and threshold the probability.
ProjectbetaTypeScriptpytest plugin for semantic assertions on LLM output, asking Jev whether plain-English claims hold with a 0.8 probability bar.
PluginbetaPythonLearning lab for DeepEval with Jev as the judge: replayed evaluations, adjustable decision rules, release gates and drift alerts.
ProjectprototypePython