Jev × NASA Kepler
Retrospective test of Jev 1.13 classifying 8,054 Kepler Objects of Interest against hidden NASA Exoplanet Archive dispositions.
This summary is based on how curated lists describe the project; its README was not available to review.
Overview
Jev x NASA Kepler is a gist documenting an independent retrospective test of Jev 1.13 on 8,054 historical Kepler Objects of Interest. The NASA Exoplanet Archive dispositions were hidden while Jev made its predictions, then used as the reference.
According to the list summary, Jev matched the archive disposition 72.5% of the time, against 64.4% for a fixed three-rule baseline, and the gist includes the exact requests, metrics, the baseline definition and caveats. It is an example of using a decision model to triage scientific records rather than replace domain analysis. No README was fetched, so details beyond the summary are not described here.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | What is the archive disposition of this Kepler Object of Interest?(inferred) | — |
How it works
- Assemble KOI records with dispositions hidden
- Ask Jev for a disposition per object
- Compare with archive dispositions
- Compare against a fixed three-rule baseline
Strengths
- Large sample of real scientific records
- Includes a simple rule baseline for context
- Exact requests published for reproduction
Caveats
- Retrospective test; archive labels are themselves the result of human vetting
- Only a summary was available here
- Single model version
At a glance
- Type
- Benchmark
- Status
- research
- Runtime
- Gist with scripts and results
Use cases and integrations
How curated lists describe it
Independent retrospective test of Jev 1.13 on 8,054 historical Kepler Objects of Interest with NASA Exoplanet Archive dispositions hidden during prediction; 72.5% archive-disposition match vs 64.4% for a fixed 3-rule baseline, with exact requests, metrics, baseline, and caveats
AnotiaWang/awesome-decision-models
Frequently asked questions
How did Jev do on Kepler objects?
The summary reports a 72.5% match with archive dispositions versus 64.4% for a three-rule baseline.
How many objects were tested?
8,054 historical Kepler Objects of Interest.
Were the labels visible to the model?
No. Dispositions were hidden during prediction.
Related projects
Experiment screening ADHD review abstracts as include or exclude with Jev, scored against Cohen et al. 2006 abstract triage labels.
AppresearchPython- jev-arena★ 122
Side-by-side tool comparing Jev and DeepSeek on labelling the same comments for relevance, sentiment and intent, with replay and reports.
ProjectbetaTypeScript Evaluation suite for decision models with 23,900 pinned rows across nine task families and three primitives, plus a leaderboard.
ProjectbetaPythonOpen-source BYOK arena where Jev faces another model as a judge on your question, with blind voting before costs are revealed.
ProjectbetaTypeScriptpytest-style tool that pins fixtures and expected Jev answers, then fails a model upgrade when answers flip or confidence drops.
ProjectbetaPythonTest harness for any Jev-API decision model: accuracy, calibration and robustness probes for reordering, injection and padding.
ServicebetaPython