jev-music-theory-1
Experiment testing whether Jev knows music theory via SATB part-writing and quizzes vs Claude, plus a Jev-driven Rhodes piano demo.
Overview
Jev Chorale Lab is a one-day experiment in two parts, with a live demo. First, Jev was given undergraduate SATB part-writing exercises through a controller that picks a voice, measure, beat, pitch, octave and duration, with no musical hints from code and a rule-based grader marking results. Score formats, prompt wording, strategies and framings were varied, and a generated theory quiz compared Jev with Claude Haiku 4.5, Sonnet 5 and Opus 5 on accuracy and cost. Second, Jev plays an endless stream of Rhodes chords by making three decisions per chord: root, quality and bass tone.
Findings reported: Jev's music theory is lexical; it knows vocabulary but cannot compute relations between spelled pitches or represent motion between chords. Framing with neighbouring beats halved errors, while score format did not matter. Its characteristic failure is a fixed point where it repeats or oscillates. Per step, Sonnet 5 made a third of Jev's errors at about 50 times the cost. The repository states it was produced autonomously by Claude Code.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Which pitch should this voice take at this beat?(inferred) | — |
| Choice | What is the next chord's root? | — |
| Choice | What quality should the chord have? | — |
How it works
- Present an exercise state in a chosen format and framing
- Jev picks controller choices step by step
- A rule-based grader scores the result
- Compare accuracy and cost with Claude models
- Separately, generate chord streams from three Jev choices per chord
Strengths
- Clear, honest description of where a decision model fails
- Cost and accuracy comparison with LLMs
- Playable live demo
Caveats
- One-day, AI-generated experiment
- Narrow domain; not general evidence about Jev
- Piano demo has no scientific value per the README
Getting started
npm install, then follow the Run it section of the README.At a glance
- Type
- App
- Status
- research
- Language
- JavaScript
- Runtime
- Web app
- Has code
- yes
Use cases and integrations
How curated lists describe it
Explores Jev on harmony exercises and music-theory questions, alongside a piano demo driven by chord choices.
logicrw/awesome-jev-projects
Frequently asked questions
Does Jev know music theory?
The write-up finds its knowledge is lexical: it knows terms but cannot compute pitch relations or voice leading.
How does it compare with Claude?
Per step, Sonnet 5 made a third of Jev's errors at about 50 times the cost; Haiku 4.5 was roughly equal.
What is the piano demo?
Jev chooses root, quality and bass tone for an endless stream of Rhodes chords that code voices and arpeggiates.
Related projects
Video demo classifying 50 mock YouTube comments with Jev (intent, value, video idea) and comparing speed and cost with GPT-4o-mini.
ModelprototypePythonAuto-clipper for talky videos: code enumerates every possible cut point and Jev judges where each standalone clip starts and ends.
BenchmarkbetaPython- jev-arena★ 122
Side-by-side tool comparing Jev and DeepSeek on labelling the same comments for relevance, sentiment and intent, with replay and reports.
ProjectbetaTypeScript Evaluation suite for decision models with 23,900 pinned rows across nine task families and three primitives, plus a leaderboard.
ProjectbetaPythonOpen-source BYOK arena where Jev faces another model as a judge on your question, with blind voting before costs are revealed.
ProjectbetaTypeScriptpytest-style tool that pins fixtures and expected Jev answers, then fails a model upgrade when answers flip or confidence drops.
ProjectbetaPython