Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up

jev-playground

Local single-page playground for Jev with templates from 37 benchmark datasets, showing gold answers next to typed results.

Appprototype
View on GitHub ↗

Overview

This Jev Playground is a local single-page app, documented in Chinese, for writing questions and calling TypeSafe System One live to see typed answers. A question bank draws real items from 37 benchmark datasets plus probes, loaded offline from a local Hugging Face cache, and shows the gold answer beside Jev's result.

Built-in templates cover sentiment (IMDB/SST-2), news topics (AG News), natural language inference (ANLI), MMLU, SMS spam, grounding checks (LLM-AggreFact), semantic similarity (STS-B), language identification across 20 languages and a multi-label emotion fan-out, each annotated with accuracy figures from a 2026 arXiv paper whose question wording is reused verbatim; the author reports reproducing all 37 results locally. The page explains how to read Choice confidence, Noul probabilities and Score positions. A standard-library Python server forwards requests using the TypeSafe SDK.

What the decision model decides

TypeQuestionAnswers
ChoiceWhich sentiment / topic / NLI label fits this text?—
Yes / noIs this SMS spam? / Is this claim grounded in the document?yes · no
ScoreHow similar are these two sentences (6 levels)?—

How it works

  1. Start the local server and open the page
  2. Pick a template or draw a real benchmark item
  3. Send it to Jev
  4. Compare the typed result with the gold answer

Strengths

  • Benchmark items available offline without spending tokens to load
  • Uses the paper's exact question wording
  • No framework or CDN; works offline apart from API calls

Caveats

  • Paths and setup are specific to the author's Windows machine
  • Chinese-language interface and docs
  • Requires a TypeSafe key

Getting started

Set TYPESAFE_API_KEY, run python server.py, then open http://127.0.0.1:8770

At a glance

Type
App
Status
prototype
Language
Python
Runtime
Local server plus static web page
Has code
yes

Use cases and integrations

How curated lists describe it

Frequently asked questions

Where do the questions come from?

From 37 benchmark datasets cached locally, using the question wording from a 2026 Jev evaluation paper.

Does loading items cost tokens?

No. Items come from a local cache; only sending to Jev costs tokens.

What does Choice confidence mean?

How concentrated the distribution is, not accuracy, as the README notes.

Related projects

  • warped-sys1-lab

    Local SvelteKit workbench and CLI for organizing, editing, running and comparing Jev experiments defined as JSON.

    CLIbetaTypeScript
  • OpenJev (GPT-AGI)

    Jev-compatible open System One engine with a Claude Code-style terminal REPL showing live probabilities, plus an experiment harness.

    ProjectprototypePython
  • jev-behavior-study

    Independent field guide to Jev 1.13.0 behaviour across 11,621 requests: framing sensitivity, Snake and 3D city driving experiments.

    ProjectresearchPython, TypeScript
  • jev-report

    Independent 52-page Chinese report on Jev that checks vendor claims and reproduces 50 Chinese tests, with traceable data.

    ProjectresearchPython
  • jev-playground (Ahmar004)

    Interactive teaching site where Jev races frontier LLMs in games, an arena, a sandbox and quizzes, with a keyless replay mode.

    AppproductionTypeScript
  • typesafe-showcase★ 1

    Next.js site with six live Jev demos, each compared side by side with Gemini or Claude on latency, cost and agreement.

    DemobetaTypeScript