mcp-server-macos-use
A Swift MCP server that lets an agent drive any macOS app: click, type and read the interface tree
- 355 ★ on the repository
- Last commit
Broad access
Why this level
- The server clicks and types in any app, so the agent can send an email, delete a file or press a pay button
- Accessibility and screen recording permissions allow reading window contents, including passwords on screen
- Every response saves a screenshot and the full interface tree to disk
Install
- Works in
Manual install
swift build -c releaseRun in a clone of the repository on macOS 13 or newer. Launch with .build/release/mcp-server-macos-use; the server speaks stdio.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The server clicks and types in any app, so the agent can send an email, delete a file or press a pay button
- Accessibility and screen recording permissions allow reading window contents, including passwords on screen
- Every response saves a screenshot and the full interface tree to disk
This is third-party code. Review the repository files before installing.
What it does
mcp-server-macos-use is a Swift MCP server that works with macOS windows through the system Accessibility APIs AXUIElement and CGEvent. It does not compare screen pixels. The agent opens an app, gets the tree of interface elements and acts on it: clicking, typing and pressing keys. The server talks to the client over stdio, and a single config entry is enough for Claude Desktop and other MCP clients. There are six tools. The first, macos-use_open_application_and_traverse, opens or activates an app by name, bundle ID or path and reads its interface. The rest take a process ID. A click is set by coordinates or by a word from the element label plus a role such as AXButton, and double and right clicks are flags. One call can click, type text and press a key. Separate tools type text, press a key with modifiers and scroll a window, and the last one refreshes the traversal without an action. Every response carries a short summary and the path to a file with the full accessibility tree and a screenshot. The agent finds the element it needs in that file with Grep and Read, so the whole screen does not land in context. Optional parameters traverse the tree before and after an action and return only the changes, and the traversal can be limited to visible elements.
Who it is for. For developers and Mac users who want Claude Code, Cursor or Claude Desktop to drive ordinary Mac apps.
Good fit when
- You need to automate an app that has no API or scripting, such as system settings or a third-party program
- The agent must find a button by its label: Submit, Open, Save
- You want to open an app, fill a form and press Return in a single call
- You build an interface check where the state change before and after a click matters
- You want the agent to search a text tree for elements without sending whole screenshots to the model
Not a fit when
- You are on Windows or Linux: the server works on macOS 13 and newer only
- The target app has a proper API or MCP server: a direct integration is more reliable than clicking the interface
- You need a web browser with a DOM: browser tools suit pages better, while this server sees them as a plain window
- You cannot grant Accessibility permissions: without them the server cannot read the tree or click
Example request
Open Calculator, work out 125 times 8 and tell me what the screen showsMCP
- Transport
- stdio
- Authentication
- not required
Limitations
- macOS 13 or newer only; building with swift build needs Swift 5.9 and Xcode Command Line Tools.
- The licenses in the repository disagree. The LICENSE file is Business Source License 1.1: production use is allowed only for non-commercial and educational purposes, and on April 9, 2028 the code switches to MIT. package.json says MIT, so check the terms with the author before commercial use.
- Clicks and key presses go through CGEvent into the system event stream, and the app cannot tell them from a live user's input.
How to disable. Remove the mcp-server-macos-use entry from the client config, for example claude_desktop_config.json, and restart the client. Then remove the app from the Accessibility list in system settings to withdraw the interface control permission. The built binary sits in .build and can be deleted together with the project folder.
FAQ
How do I connect mcp-server-macos-use to Claude Desktop?
Build the server with `swift build -c release` and add an mcpServers block to claude_desktop_config.json with the key mcp-server-macos-use and a command field holding the absolute path to the built file. Then restart Claude Desktop.
How does the server read an app's interface?
Through the macOS Accessibility APIs: AXUIElement for elements and CGEvent for actions. The server builds a tree of elements and does not recognise an image. Every response includes the path to a file with the tree and a screenshot, and the agent searches that file for the element it needs.
How do I click a button by its label instead of coordinates?
In macos-use_click_and_traverse pass element with a part of the label, case-insensitive, such as Open, instead of x and y, and a role such as AXButton if needed. The doubleClick and rightClick flags cover double and right clicks.
Do I need the screen recording permission?
Reading the tree needs the Accessibility permission. A screenshot comes with every response, so in practice screen recording may be needed too. Grant the permissions to the app that launches the server.
Can I use the server in a commercial project?
Only with caveats. The LICENSE file is BSL 1.1, which allows production use only for non-commercial and educational purposes until the switch to MIT on April 9, 2028. package.json names MIT, so write to the author at matt@mediar.ai before a commercial launch.
Discussion
No questions or reviews yet.
Related
MCP serversAn MCP server for macOS: the agent moves the mouse, types, manages windows and captures the screen when an app has no API
A macOS CLI and MCP server: screenshots, accessibility-based UI inspection and app control via clicks and typing
Open Computer Use
open-computer-use
An open MCP server for desktop control: the agent reads app state and clicks, types and scrolls on macOS, Linux and Windows
MCP server for AppleScript: the agent runs scripts and controls Mac apps, from Notes and Calendar to files
An MCP server for driving Windows: the agent opens apps, clicks and types, reads window and UI state, and works with files and PowerShell
Clawd Cursor: desktop control through MCP
Clawd Cursor
An MCP server that lets an agent drive windows and apps: it reads the screen as structure and acts by element name
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| mcp-server-macos-use (this entry) | no | not required | 355 | |
| Remote macOS Use: control of a remote Mac | no | API key | 490 | shows the screen of a remote Mac over VNC, while this server drives local apps |
| Open Computer Use | no | not required | 2,765 | an open MCP desktop-control server with a different implementation |
README in short
The main page is short and covers five tools, while the full description with the sixth scrolling tool and system requirements is in llms.txt. The author offers to tailor the server to a team's needs on request through an issue or email.