jev-typesafe-real-financial-use-cases
Decision Lab: sixty finance demos asking Jev, OpenAI Decisions and twelve other decision models the same questions, graded.
Overview
Decision Lab compares typed decision models on real financial work. Fifty demos across ten domains put the same Choice, Score and Noul questions to TypeSafe's Jev (via the official JavaScript SDK) and OpenAI's decisions API over ledgers, orders, fraud, wallets, portfolios, trades, filings and strategies, with a switch to view either model's run; ten more image demos (receipts, cheques, invoices, charts) run on OpenAI only. Each demo shows the state sent, the answers returned and a report graded against data where the right answer is known.
A benchmark page puts every model head to head, including twelve more decision models reached through OpenRouter's decisions endpoint from providers such as Microsoft, Cloudflare, Perplexity, Liquid and Together. It reports items matching ground truth with intervals, lift over a simple rule, calibration error, median and 95th-percentile latency and cost per thousand items, counted only on items every model answered. A lab backtests Jev's trade decisions on historical daily candles. The deployed site replays recorded answers, and scoring runs offline and free.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this transaction fraudulent? (example of graded finance questions)(inferred) | — |
| Choice | Which category does this ledger entry or trade belong to?(inferred) | — |
| Score | How risky is this portfolio or strategy?(inferred) | — |
How it works
- Define typed questions over financial state with known answers
- Record each model's answers once
- Grade against ground truth and a simple-rule baseline
- Publish leaderboard, per-demo matrix and trade backtests
Strengths
- Same questions across many decision models
- Ground-truth grading with intervals and calibration error
- Offline, free re-scoring from recorded runs
Caveats
- Image demos run on OpenAI only
- Recorded runs may lag model updates
Getting started
npm run score (offline re-scoring of recorded runs).At a glance
- Type
- Project
- Status
- research
- Language
- JavaScript
- Runtime
- Web site + Node scripts
- Has code
- yes
Use cases and integrations
How curated lists describe it
Fifty real-world financial use cases for TypeSafe.
wh000wh000/awesome-jev-liveFifty real-world financial use cases for TypeSafe's Jev model: typed, structured LLM answers over ledgers, fraud, portfolios, trades and filings, each graded against data where the right answer is known.
hellogumbo/awesome-jev
Frequently asked questions
Which models are compared?
Jev, OpenAI's decisions API and twelve more decision models via OpenRouter.
How are answers graded?
Against data where the right answer is known, with intervals, lift over a simple rule and calibration error.
Does viewing the site call the APIs?
No, the deployed site replays recorded answers.
Related projects
PyTorch factor-research stack with 213 factors and cost-aware backtests, plus a preview typed portfolio-decision adapter.
ProjectbetaPythonTrade Arena proof of concept comparing trading strategies on 13 US stock funds, grown out of a pre-registered Jev study.
ProjectprototypePythonEVM token screener that returns ape, watch or avoid with probabilities and risk scores, via library, CLI exit codes or MCP.
MCP serverprototypeJavaScriptReproducible test of viral Jev HFT claims: direction not calibrated, never under 100 ms, and no edge after trading fees.
ServiceresearchPythonPoint-in-time O'Neil momentum backtest where Jev picks entries and judges exits, with a jackknife showing no resolvable edge over rules.
ProjectresearchPythonBacktest harness that tests Jev as a BUY/SELL/HOLD trader on NQ level-10 order-book data, with replay, viewer and paper-trading modes.
ProjectresearchPython