macos-computer-use-kit
Accessibility-first macOS computer-use MCP server and CLI, with optional Jev guards on targets and inputs before irreversible actions.
Overview
macos-computer-use-kit gives AI agents macOS computer use through the accessibility tree instead of guessing coordinates from screenshots. It runs as a stdio MCP server and CLI for Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, opencode, pi and DeepSeek Harness. Each element has a stable reference, role, label and geometry, re-observing returns only the diff, input is posted to the target app in the background so the user's cursor never moves, and every result separates 'action sent' from 'verified' with retry guidance.
It is safe by default, with deny lists for password managers and system auth, refusal to type into secure password fields, blocked lock and log-out chords, dry-run mode and audit logs, and uses set-of-mark screenshots or on-device OCR only as fallbacks. Optional Jev semantic guards add calibrated judgments about the target and input before an irreversible action, with the decision itself kept in code. It ships as a pip package, a pi package and a DeepSeek Harness plugin, with Chinese documentation.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | Is this the intended target element for the irreversible action?(inferred) | — |
| Yes / no | Is this input appropriate to send to this target?(inferred) | — |
How it works
- Agent snapshots the accessibility tree
- Agent acts on an element reference
- Optional Jev guard judges target and input before irreversible actions
- Code decides whether to proceed
- Action is posted in the background and verified by diff
Strengths
- No cursor hijacking
- Verified actions with retry guidance
- Strong default safety rules
- Works with many agent hosts
Caveats
- macOS only
- Jev guards optional and require a key
- Canvas and custom UIs need vision fallbacks
Getting started
pip install macos-computer-use-kit and register the MCP server with your agent (see Quick start).At a glance
- Type
- Plugin
- Status
- beta
- Language
- Python
- Runtime
- MCP server and CLI on macOS
- Has code
- yes
- Has tests
- yes
Use cases and integrations
How curated lists describe it
AX-first computer use for AI agents on macOS with optional Jev (TypeSafe System One) semantic guards: calibrated target/input judgments before an irreversible action, decisions kept in code. Accessibility-tree targeting, window-scoped input, clipboard-safe paste, read-back verification. Ships a pip CLI, a pi package and a DeepSeek Harness plugin.
hellogumbo/awesome-jev
Frequently asked questions
Does it use screenshots?
Mainly the accessibility tree; screenshots and OCR are fallbacks.
Where does Jev come in?
As optional semantic guards before irreversible actions; code makes the final decision.
Will it move my mouse?
No, input is posted to the target app in the background.
Related projects
Codex computer-use skill where Jev picks the next on-screen element, action, completion and risk from text candidates.
Agent skillprototypeJavaScriptZero-dependency Node library with MCP and ACP adapters adding Jev decisions and gated computer use beside any LLM agent.
MCP serverbetaTypeScriptMCP server giving coding agents fast typed decisions (ship gates, risk checks, file triage, page checks) as accept, review or escalate.
MCP serverbetaTypeScriptMCP server that lets agents observe and control Android phones over ADB, with Jev picking controls and judging goal completion.
MCP serverbetaTypeScriptmacOS computer-use package: a background app driver for any agent, plus a loop where Jev picks each UI action for a bounded subtask.
ProjectbetaPythonWindows MCP server and CLI where Jev observes windows and chooses controls so agents can hand off whole desktop tasks.
MCP serverbetaPython