jev-use (coco-research)
Pre-alpha Rust and Tauri voice control layer for macOS that reads the accessibility tree; Jev is a fallback rather than the router.
Overview
jev-use (Coco Research) is a voice-driven control layer for macOS: speak a goal and the Mac carries it out, with everything running locally. It observes the screen through the accessibility tree and is designed to drive it. The pipeline decides whether speech is a command or dictation, resolves which capability fulfils a command through a registry in about 50 ms without a model, observes the screen, and then acts. A listing summarises the design as Jev being the fallback, not the router.
It is pre-alpha with no release. Observation is done and at parity with a reference implementation on Finder, Terminal, an Electron app and Notes, proven by a parity harness; the voice hook is in daily use, pasting dictation in 0.24 to 0.33 seconds and handing off commands in 6 to 15 ms. Execution, the CLI and the app UI are not yet built. The layers are deliberately separated so the screen layer knows nothing about models. The repository also carries the organisation's Repo Standard. It requires macOS 26 and Rust, and has 77 workspace tests.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Choice | Is this utterance a command or dictation?(inferred) | command · dictation |
| Choice | Which on-screen element should be acted on when no registered capability matches (fallback)?(inferred) | — |
How it works
- Voice hotkey captures speech
- Decide command versus dictation
- Resolve via a capability registry; fall back to a decision model when needed
- Observe the accessibility tree and act (execution not yet built)
Strengths
- Local-first design
- Measured fast voice handling
- Parity harness proves observation matches the reference
Caveats
- Pre-alpha; execution not implemented
- No release or installer
- macOS 26 only
Getting started
cargo test --workspace, then `cargo run --example walk -p jev-ax -- --app Finder --json`.At a glance
- Type
- Project
- Status
- prototype
- Language
- Rust
- Runtime
- macOS (Tauri)
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
Voice-driven control layer for macOS. Jev is the fallback, not the router. Rust core, Tauri shell.
hellogumbo/awesome-jev
Frequently asked questions
Does it work end to end yet?
No. Observation and the voice hook work; executing actions is still to be built.
What role does Jev play?
Listings describe it as a fallback; a model-free capability registry resolves most commands first.
What does it need?
macOS 26, a Rust toolchain and the Accessibility permission.
Related projects
macOS menu-bar Swift app that turns on-device transcribed speech into Jev typed decisions and macOS actions, confirming hard-to-undo steps.
AppbetaSwiftVoice assistant daemon for Hyprland Linux desktops that acts through tool modules, with Jev gating which requests and actions proceed.
ProjectbetaPythonTalk to your Mac: local whisper.cpp transcribes, one Jev request picks the action and typed arguments, and code executes macOS automation.
ProjectprototypePythonFloating macOS bar for voice or text commands where Jev picks the next action from live accessibility controls in a loop.
ProjectprototypeSwiftmacOS voice and typed computer-use app where Jev picks the next on-screen action from the Accessibility tree, with no screenshots.
AppprototypeSwiftElectron app for controlling Windows 10/11 by Turkish or English speech, using OpenAI Realtime, Jev decisions, UI Automation and Playwright.
ProjectprototypeTypeScript