Skip to content

Hermes JIT Context OS

JIT-JEV Context OS: deterministic context compiler plus a Jev System 1 gate that scores AST relevance, test proofs and tool targets.

BenchmarkresearchMIT

Overview

JIT-JEV Context OS is an epistemic context runtime for autonomous coding agents with real tool access. It combines a deterministic JIT context compiler, which builds a dense context capsule instead of stuffing tens of thousands of tokens of logs and source into the prompt, with a Jev System 1 gate and domain router. According to a list summary, Jev scores AST relevance, test proofs and tool targets, with fail-open circuit-breaker resilience.

The README headline reports large gains on a commercial 45-module codebase (fewer turns, faster delivery and far smaller prompts) and a 31.3% reduction in turns per task attributed to the Jev layer. However, a more detailed section is candid that in another evaluation almost all of the gain came from deterministic harness fixes such as tool declarations, a loop guard, a forced last-turn answer and a real turn budget, that the judge added a few tasks within sampling noise at higher time per task, and that a cognitive pre-pipeline did not help and is being removed.

What the decision model decides

TypeQuestionAnswers
ScoreHow relevant is this AST node or file to the task?(inferred)—
Yes / noDoes this test result prove the fix?(inferred)yes · no
ChoiceWhich tool target should the agent use next?(inferred)—

How it works

  1. Compile a compact context capsule deterministically
  2. Jev gates and routes: relevance, proofs, tool targets
  3. Agent executes tools; physical feedback (exit codes, assertions) arbitrates
  4. Circuit breaker fails open if Jev is unavailable

Strengths

  • Much smaller prompts via deterministic compilation
  • Jev gate optional with fail-open behaviour
  • Includes candid analysis of where gains came from

Caveats

  • Headline numbers come from limited runs
  • Detailed results attribute most gains to deterministic fixes
  • Promotional framing

Getting started

Create a Python venv, pip install -r requirements.txt, and follow the README to attach the runtime to a coding agent.

At a glance

Type
Benchmark
Status
research
Language
Python
Runtime
agent runtime / harness layer
Has code
yes
License
MIT

Use cases and integrations

How curated lists describe it

  • Coding agents: uses Jev as a sub-millisecond System 1 Epistemic Gate and Domain Router to score AST relevance, test proofs, and tool targets, cutting autonomous agent turns by 31.3% and blind file exploration by 52.6% on SWE-bench with fail-open circuit-breaker resilience.

    yibie/awesome-jev

Frequently asked questions

What does Jev do here?

Scores relevance, test proofs and tool targets as a System 1 gate.

Where do the gains come from?

The README's detailed section attributes most gains to deterministic harness fixes.

What if Jev fails?

A circuit breaker fails open.

Related projects

  • Agent plugin that asks Jev whether the session is at a safe boundary to /compact and can trigger it on Pi and Claude Code.

    PluginbetaTypeScript
  • Python agent runtime where Jev Choice decisions pick tools and arguments, and low-confidence picks go to an LLM for review.

    ServicebetaPython
  • Pi coding agent extensions, including pi-jev-router, which asks Jev whether a prompt should continue, fork or start a new session.

    PluginbetaTypeScript
  • Experiment using Jev Noul judgments on accumulated evidence to decide when Zilliz's DeepSearcher agent should stop searching.

    ProjectresearchPython
  • Research code evaluating Jev as an independent per-tool relevance judge for building task-level tool spaces on TGB, BFCL and GTA.

    ServiceresearchPython
  • Deterministic evidence selector for noisy Codex investigations; its earlier Jev subagent router is kept for the record, not recommended.

    Projectresearch