jev-demos (mani-aiml)
Video demo code comparing Jev with Laya, Strands Decider and Claude on grading, fetching and prompt-injection resistance.
Overview
These demos accompany videos on The Agentic Enterprise YouTube channel and compare Jev with other System One models and LLMs. One folder runs eight tasks with Claude alone versus Jev fetching and Claude writing. Another has Jev and Laya grade the same 500 AI-written answers zero-shot against a reference judge, including cascade maths. A third asks whether a decision model can be prompt-injected, testing Jev and Claude Haiku 4.5 as a review gate on hand-written attack emails and real attacks from Microsoft's LLMail-Inject.
Further folders fine-tune Laya on the grading job and score it against Jev on the same items, and run Strands Decider 2B out of the box and fine-tuned on the same split. Each folder is self-contained with its own requirements and keys. Data is synthetic except the LLMail sample, which is fetched from the source; fine-tuned checkpoints are not included.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this AI-written answer acceptable?(inferred) | — |
| Yes / no | Does this email contain a prompt-injection attempt?(inferred) | — |
| Choice | Which lookup should be fetched for this task?(inferred) | — |
How it works
- Choose a demo folder and install its requirements
- Add the keys that folder names
- Run the notebook comparing Jev with Laya, Strands Decider or Claude
- Review recorded results and cascade or injection analysis
Strengths
- Head-to-head comparisons on identical items
- Includes prompt-injection testing of decision models
- Fine-tuning pipelines with data checks
Caveats
- Checkpoints not included
- Synthetic data except the LLMail sample
- Results tied to specific videos
Getting started
In a demo folder: `python3 -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt`, then copy .env.example to .env with the named keys.At a glance
- Type
- App
- Status
- research
- Language
- Python
- Runtime
- Jupyter notebooks
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Demos with Jev, TypeSafe.
wh000wh000/awesome-jev-liveDemos with Jev, TypeSafe's System One model, one folder per demo. Code behind the videos on The Agentic Enterprise.
hellogumbo/awesome-jev
Frequently asked questions
What is compared?
Jev, Laya, Strands Decider and Claude on grading, fetching and injection resistance.
Can a decision model be prompt-injected?
One demo tests exactly that with hand-written and real LLMail-Inject attacks.
Are model weights provided?
No, fine-tuned checkpoints are not included.
Related projects
Evaluation suite for decision models with 23,900 pinned rows across nine task families and three primitives, plus a leaderboard.
ProjectbetaPythonBenchmark harness comparing Jev with OpenRouter models and open Jev-shaped models on accuracy, calibration, latency and cost.
ProjectresearchPythonShort report comparing Jev and many other fast decision classifiers with an LLM on grading section relevance in technical documents.
ProjectresearchOpenlayer library packing agent evals and guardrails (tool choice, grounding, injection, PHI) into one decision-model request.
BenchmarkbetaPythonLarge rubric-classification benchmark of Jev vs Claude Haiku 4.5, Sonnet 5, OpenJev and many open models, with full price tracking.
BenchmarkresearchPythonOpenBles Firewall 1: open prompt-injection detector for agent inputs, with a Claude Code hook and an optional Jev backend.
MCP serverbetaPython