can-jev-play
Experiments testing whether Jev can spot positive-expected-value dice bets from payout tables, and whether recent wins or losses sway it.
Overview
Can Jev Play tests AI decisions in games where the correct move is known. The first game is a fair six-sided die with exact payouts and one choice, Bet or Skip, judged by expected value rather than whether the roll wins.
The latest experiment collected 12,000 decisions from 3,000 payout tables on jev-1.13.0 under four history conditions. Jev took most profitable bets but struggled to reject unprofitable ones: with no history it bet 96.0% of the time, skipping only 6.3% of negative-EV offers, for 52.2% overall correctness. Recent wins reduced its willingness to bet even though rolls are independent. A Noul variant also bet on almost everything. The README notes that question wording changed with the primitive, so question type alone is not isolated. Data and individual requests are explorable.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Bet or skip this dice offer? | Bet · Skip |
| Yes / no | Is this offer positive expected value? | yes · no |
How it works
- Generate payout tables with known EV
- Add a history condition (mixed, wins, losses, none)
- Ask Jev to bet or skip
- Score against EV, not outcomes
Strengths
- Ground truth by construction
- Matched sets isolate history effects
- Raw data published
Caveats
- Single game so far
- Choice vs Noul comparison confounded by wording
At a glance
- Type
- Project
- Status
- research
- Runtime
- web explorer + experiment scripts
- Has code
- yes
Use cases and integrations
How curated lists describe it
Can Jev infer whether a bet is worthwhile from its payout table, and do recent wins or losses sway that choice.
hellogumbo/awesome-jev
Frequently asked questions
Can Jev spot bad bets?
Poorly; with no history it skipped only 6.3% of negative-EV offers.
Does history matter?
Yes, recent wins made it bet less despite independent rolls.
How many decisions?
12,000 from 3,000 payout tables.
Related projects
pytest-style tool that pins fixtures and expected Jev answers, then fails a model upgrade when answers flip or confidence drops.
ProjectbetaPythonTest harness for any Jev-API decision model: accuracy, calibration and robustness probes for reordering, injection and padding.
ServicebetaPythonZero-shot run of Jev 1.13 on all seven LexGLUE legal tasks (23,607 examples) versus GPT-5.6 Luna, open models and fine-tuned BERT.
ProjectresearchPythonPre-specified study running Jev and GPT-4.1 as 300 synthetic survey respondents; question format mattered more than model.
ProjectresearchPythonPre-registered paired audit of Jev on Spanish across 3,200 labelled items, plus a CLI to rerun it on your own data.
CLIresearchPythonPre-registered independent evaluation of Jev on intent classification and confidence-gated cascades, with a transparent erratum.
LibraryresearchPython