Hafen
Desktop tool for testing chat endpoints conversationally, with rule checks and judged checks including TypeSafe's System One.
Overview
Hafen tests chat endpoints the way users talk to them. You send messages to any chat API and watch replies stream as a conversation, with thinking, tool calls, text, timings and a badge for every check shown live, and a timeline of every step of a run. It supports plain HTTP with JSON, SSE or NDJSON, and presets for OpenAI, Anthropic, Gemini and Ollama, plus plugins and login fixtures.
Checks are either rule-based (JSONPath, status, latency, length, tokens, cost, tool calls, thinking, event order) or judged, using TypeSafe's System One, LLM judges with rubrics, or reference answers. Every result is pass, fail, unsure, skipped or error with a reason, and judged results carry a confidence. Suites are kept as plain YAML and JavaScript that export to your repository and run unchanged in CI via a CLI. It is an early proof of concept.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Does this chatbot reply satisfy the check criterion?(inferred) | pass · fail · unsure |
How it works
- Configure an endpoint and write a conversation test
- Run it and watch replies stream
- Apply rule checks and judged checks (System One or LLM judge)
- Export suites and run them in CI
Strengths
- Unsure outcome with reasons and confidence
- Same tests in app and CI
- Many endpoint types supported
Caveats
- Proof of concept
- System One is one of several judge options
Getting started
Download the desktop app per the README, configure an endpoint and start a suite; export to run via the CLI in CI.At a glance
- Type
- Project
- Status
- prototype
- Language
- TypeScript
- Runtime
- desktop app + CLI
- Has code
- yes
Use cases and integrations
How curated lists describe it
A clean and minimalist desktop tool to test your AI agents the way your users talk to them.
yibie/awesome-jev
Frequently asked questions
How does Hafen use System One?
As one of the judged check types for chatbot replies.
Can tests run in CI?
Yes, exported suites run unchanged with the CLI.
Is it stable?
No, it is a proof of concept.
Related projects
Semantic test matchers for Vitest and Jest that ask Jev whether output satisfies a plain-English expectation and threshold the probability.
ProjectbetaTypeScriptpytest plugin for semantic assertions on LLM output, asking Jev whether plain-English claims hold with a 0.8 probability bar.
PluginbetaPythonBenchmark for agent regression after config edits: a Jev Choice decides which approved answers changed meaning versus wording.
BenchmarkresearchPythonTool where Jev clicks through your web app from plain-language steps, stops below 80% confidence, and a vision model checks pages.
Agent skillprototypeJavaScriptAgent observability platform that finds and fixes agent failures, with an optional Jev preclassifier for conversation checks.
ProjectproductionTypeScriptOpen-source Claude Cowork alternative built on OpenCode; per list notes, its eval testkit uses Jev as a verification judge.
ProjectproductionTypeScript