agent-desktop

What it does

Computer use for AI agents from Vercel Labs: it lets an agent see and operate desktop apps on macOS, Windows and Linux through the operating system's accessibility tree instead of screenshots and pixel matching. snapshot returns an app's UI as JSON with a ref for each element, the agent clicks or types on a ref, then takes a new snapshot to check the result. There is no model inside; the loop runs in your agent. It ships as a CLI and a C library, and agent-desktop mcp serves every command as an MCP tool over stdio. Apache-2.0 licensed.

Who it's for

People who want an agent to work with native apps (Finder, Slack, system settings and so on) and not just the browser, and are comfortable giving it accessibility access to their desktop.

How to set it up

1. Install the CLI globally (it needs Node 18 or later; Homebrew also works on macOS and Linux with brew install agent-desktop), then check what your platform supports:

   bashCopy

   npm install -g agent-desktop
   agent-desktop status

2. macOS: give the app that launches agent-desktop (Terminal, iTerm2, VS Code or your agent's app) Accessibility permission. This command opens System Settings at the missing permission. Screen Recording is only needed for screenshots:

   bashCopy

   agent-desktop permissions --guide

3. Try the loop. Refs belong to the snapshot they came from, so take a new snapshot after the UI changes:

   bashCopy

   agent-desktop snapshot --app Finder -i
   agent-desktop click @s8f3k2p9:e8
   agent-desktop snapshot --app Finder -i

4. To use it as an MCP server, add this to your client. Tools are named after the commands, such as desktop_snapshot and desktop_click:

   jsonCopy

   {
     "mcpServers": {
       "agent-desktop": {
         "command": "agent-desktop",
         "args": ["mcp"]
       }
     }
   }

5. Or install the skill for a shell-based agent: npx skills add https://github.com/vercel-labs/agent-desktop --skill agent-desktop. Requirements: macOS 13 or later, Windows 10 1809 or later, or Ubuntu 24.04 or later with a running AT-SPI accessibility bus. Ref actions don't move the pointer unless you pass --headed. Snapshots, clipboard reads and screenshots can contain private content from your desktop, so keep them out of public logs.

Check the official agent-desktop docs for the latest steps.

Install agent-desktop in your client

Runs on your computer. Needs: The agent-desktop CLI installed globally (npm install -g agent-desktop, Node 18 or later); macOS needs Accessibility permission for the launching app. Sign-in or keys: None.

claude mcp add --transport stdio agent-desktop -- agent-desktop mcp

{
  "mcpServers": {
    "agent-desktop": {
      "command": "agent-desktop",
      "args": [
        "mcp"
      ]
    }
  }
}

{
  "mcpServers": {
    "agent-desktop": {
      "command": "agent-desktop",
      "args": [
        "mcp"
      ]
    }
  }
}

{
  "servers": {
    "agent-desktop": {
      "type": "stdio",
      "command": "agent-desktop",
      "args": [
        "mcp"
      ]
    }
  }
}

codex mcp add agent-desktop -- agent-desktop mcp

[mcp_servers.agent-desktop]
command = "agent-desktop"
args = ["mcp"]

• If the server shows as failed or disconnected, see fixing MCP error -32000.
• Compare every server's runtime and sign-in needs in the MCP server requirements table.

Checked Nov 11, 2026 against the official docs. Snippets use placeholders; keep real keys out of files you commit. Windsurf is not listed because we have not verified its format.

Set it up / Fixes

Step-by-step guides for getting agent-desktop running and fixing common errors.

• Setup:How to add an MCP server to Cursor
• Setup:The best MCP servers for Claude Code
• Fix:Fix MCP error -32000: "Connection closed"

Example prompts

Use agent-desktop to open System Settings and tell me which display resolution is selected.

Take a snapshot of the Finder window, open the Downloads folder and list the five newest files.

Open my app's desktop build, click through the onboarding screens and report any button that doesn't respond.

Made agent-desktop? Add the badge

If you like, show that agent-desktop is listed and hand-checked here. It's optional: listings are free and we never ask for a link back.

[![Hand-checked on PluginStack](https://www.pluginstackai.com/badge/hand-checked.svg)](https://www.pluginstackai.com/plugins/agent-desktop/)

<a href="https://www.pluginstackai.com/plugins/agent-desktop/"><img src="https://www.pluginstackai.com/badge/hand-checked.svg" alt="Hand-checked on PluginStack" width="226" height="28"></a>

About the badge

Free: MCP Starter Checklist

After you set this tool up, use the one-page checklist so your next MCP server doesn't fail on first connect.