OpenComputerUse

Rust MCP server for background computer use; plain-English recipe steps are placed by one Jev or Clef choice over interactive elements.

MCP serverbeta

View on GitHub ↗

Overview

OpenComputerUse is an MCP server, written in Rust, that lets agents drive desktop apps in the background. You start a session with an app and get a session id; the app works behind your other windows while your pointer, keyboard focus and frontmost app stay put, and sessions end when the server exits. Tools cover sessions and windows, screenshots of covered windows with optional accessibility trees, element-indexed UI trees, pointer and keyboard actions, setting values, element actions and file-picker answers. It ships for macOS via Homebrew or DMG and as single binaries for Linux and Windows.

Per the list summary, plain-English recipe steps are placed by one Jev (or Clef) choice over the window's interactive elements, and execution stops when the model is not confident. The fetched README excerpt does not cover this feature in detail, so the exact question is inferred.

What the decision model decides

Type	Question	Answers
Choice	Which interactive element does this recipe step refer to?(inferred)	—

How it works

Start a background session with an app.
Read the window's interactive elements.
Ask Jev or Clef to choose the element for a plain-English step.
Act if confident; otherwise stop.

Strengths

Does not steal pointer or focus.
Bounded element choice with a confidence stop.
Cross-platform binaries.

Caveats

Jev/Clef recipe placement known only from the list summary.
Third-party Homebrew tap must be trusted.

Getting started

brew tap IAmJSD/OpenComputerUse https://github.com/IAmJSD/OpenComputerUse && brew trust iamjsd/opencomputeruse && brew install --cask opencomputeruse, then add opencomputeruse mcp to your MCP client.

At a glance

Type: MCP server
Status: beta
Language: Rust
Runtime: MCP server, desktop app
Has code: yes

Use cases and integrations

Agent Decisions340

How curated lists describe it

Background computer use for agents as an MCP server, written in Rust: sessions drive an app behind your other windows, and plain-English recipe steps are placed by one Jev (or Clef) choice over the window's interactive elements, stopping when it is not confident.
MrJev/awesome-jev

Frequently asked questions

Does it take over my screen?

No, apps run in the background and your pointer and focus stay where they are.

Where does Jev fit?

Per the list summary, it places plain-English recipe steps onto UI elements and stops when unsure.

Which platforms are supported?

macOS, Linux and Windows.

Related projects

zero-use-computer
Rust MCP server for desktop control via accessibility trees, with an optional decision model (Jev or compatible) for small choices.
MCP serverproductionRust
agent-desktop
Rust CLI for desktop computer use via accessibility trees, with optional jev-desktop scripts letting Jev pick the control and action.
Agent skillbetaRust
arc-cua
macOS computer-use package: a background app driver for any agent, plus a loop where Jev picks each UI action for a bounded subtask.
ProjectbetaPython
Flick
Local stdio MCP server that executes whole browser and macOS goals with a Jev-driven observe-decide-act loop.
MCP serverbetaTypeScript
jev-adb
MCP server that lets agents observe and control Android phones over ADB, with Jev picking controls and judging goal completion.
MCP serverbetaTypeScript
jev-browser-skill (ChenYCL)
Browser and computer use for coding agents via CLI and MCP, with Jev's calibrated judgments and a code-owned control loop.
MCP serverbetaTypeScript