jev-canvas
Draw on a tldraw canvas by voice while pointing a webcam-tracked finger; Jev decides command, action, shape, colour, target and place.
Overview
jev-canvas lets you say make a yellow circle here while pointing at the screen, and the circle appears under your fingertip on a tldraw canvas. Speech comes from the browser's Web Speech API, the fingertip from MediaPipe hand tracking on the webcam, and every decision comes from Jev through OpenRouter's Decisions API. On each partial transcript Jev answers eight typed questions: is this a command, is the sentence complete, which action, which shape, which colour, which existing shape is targeted, where, and what size; plain code gates them with thresholds and executes the edit.
Commands work in English and Ukrainian, switching the recogniser between languages: create, move, delete, recolour, resize, duplicate, clear, undo and redo, across eleven shape types, the 13 tldraw colours, the pointed location, the centre and eight regions. Speech not addressed to the canvas is ignored; a remark about lunch scores is_command around 2%. The yibie list reports 300-550 ms per decision.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this utterance a command to the canvas? | yes · no |
| Choice | Which action? | create · move · delete · recolor · resize · duplicate · clear · undo · redo |
| Choice | Which shape and colour? | — |
| Choice | Where: pointed location, centre or a region? | — |
How it works
- Capture speech and fingertip position
- On each partial transcript, ask Jev eight questions
- Code applies thresholds to decide whether to act
- Execute the edit on the tldraw canvas
Strengths
- Real-time multimodal control without an LLM
- Ignores non-command speech
- Bilingual commands
Caveats
- Depends on browser speech recognition and webcam tracking
- One language at a time
At a glance
- Type
- Demo
- Status
- prototype
- Language
- TypeScript
- Runtime
- browser (tldraw, MediaPipe)
- Has code
- yes
- Stars
- 16
Use cases and integrations
How curated lists describe it
Multimodal UI: draw on a tldraw canvas by voice while pointing a webcam-tracked finger; on every partial transcript Jev answers eight typed questions (is it a command, is the sentence complete, action, shape, colour, target, place, size) and plain code gates them with thresholds, in English and Ukrainian, 300–550 ms per decision.
yibie/awesome-jevDraw on a tldraw canvas with your voice and a webcam-tracked finger; Jev decides action, target and place on every partial transcript. English and Ukrainian commands.
AnotiaWang/awesome-decision-modelsDraw on a tldraw canvas with your voice and a pointing finger. Jev (TypeSafe System One) decides action, target and place in ~350 ms per spoken word.
wh000wh000/awesome-jev-liveVoice and finger-pointing control of a tldraw canvas: Jev picks the action, target shape and place from each partial transcript plus the fingertip position; deterministic code applies thresholds and executes the edit.
cobanov/awesome-jevDraw on a tldraw canvas with your voice and a pointing finger. Jev (TypeSafe System One) decides action, target and place in ~350 ms per spoken word.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide?
Whether speech is a command, and its action, shape, colour, target, place and size.
Which languages work?
English and Ukrainian.
How fast is it?
About 300-550 ms per decision, per the list.
Related projects
Mac voice assistant where one Jev call per utterance maps speech to app, volume, Spotify and system actions; an LLM only answers questions.
AppprototypePythonExperimental voice-driven auto-advance for Slidev that asks Cloudflare's Clef-omni four typed questions when the presenter pauses.
ProjectprototypeTypeScriptOne text box that morphs into the right UI card as you type, with Jev answering 14 typed questions per input and code doing the maths.
DemoprototypeTypeScriptPage companion where a site's actions are WebMCP tools and Jev picks the action a visitor's request means, asking back when unsure.
LibraryprototypeHome Assistant conversation agent that maps Assist commands to built-in intents using a Jev fan-out of typed questions.
ProjectbetaPythonLocal web and voice interface for Codex and Claude sessions in tmux, where Jev interprets each request and Python performs it.
ProjectprototypePython