computer-use-mcp
An MCP server that lets a model drive your desktop: screenshots, mouse and keyboard through a single computer tool
- 394 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- The model gets the whole mouse and keyboard: it can open any application, read any data on the screen and change or delete files
- The author warns that models make frequent mistakes and fall for prompt injections from web pages and documents on the screen
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add mcp/computer-use-mcpDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The model gets the whole mouse and keyboard: it can open any application, read any data on the screen and change or delete files
- The author warns that models make frequent mistakes and fall for prompt injections from web pages and documents on the screen
This is third-party code. Review the repository files before installing.
What it does
computer-use-mcp adds a single `computer` tool to an MCP client, and the model uses it to work at your computer like a person. It takes a screenshot, finds the coordinates of the element it needs, moves the cursor, clicks, drags, scrolls and types text. The tool follows the scheme of Claude's official computer use tool but starts locally with one npx command. It has eleven actions: pressing a key or a combination, typing text, moving the cursor, left, right, middle and double click, dragging, scrolling by direction and pixel count, taking a screenshot and reading the cursor position. The server draws a red crosshair on the screenshot where the cursor sits, so the model can see how far it missed. Hints in the tool description nudge the model toward keyboard shortcuts over clicks. Screenshots go through nut.js, and on newer macOS versions the screencapture command is the fallback. On Linux typing goes through xdotool, which respects the keyboard layout. By default the server talks to the client over stdio, and HTTP mode is turned on with MCP_TRANSPORT=http.
Who it is for. For people who want to try desktop control from Claude Code, Claude Desktop or Cursor and are ready to watch every step of the model, ideally in a separate user account.
Good fit when
- You want to see how a model handles a task in a graphical interface, for example changing the desktop wallpaper
- The task lives in an application with no API and no command line, so only the mouse and keyboard remain
- You set an agent up on a separate virtual machine or a throwaway account and want to give it the whole screen
- You are building a trial scenario where the model clicks through the interface and you watch the screenshots
Not a fit when
- A main account with work mail, keys and money: the author compares that access to handing your computer to a hyperactive toddler
- The task has an API or a command line: that is faster and more reliable than clicking pixels
- The agent runs unattended: the author warns that models make frequent mistakes and are vulnerable to prompt injections
- The screen is large and high-resolution: the author recommends a small resolution such as 720p
Example request
Take a screenshot, open system settings and change the desktop wallpaper to any dark one, checking the result with a new screenshot after each step.MCP
- Transport
- stdio, http
- Authentication
- not required
Environment variables
MCP_TRANSPORT- The value http turns on HTTP mode instead of stdio
Limitations
The author warns: at the time of writing, models make frequent mistakes and fall for prompt injections, and the server gives them complete control of the computer. The author advises watching the work and, where possible, running the server in an isolated user account.
Better results need a model that is strong at computer use, and the author recommends the latest Claude models. A small screen resolution works better, for example 720p; on macOS the author suggests the displayoverride-mac utility. For websites the Rango extension helps, because it gives keyboard navigation that is more reliable than clicking coordinates. On macOS a click on an unfocused window may only bring it to the front, so the action sometimes has to be repeated.
The server runs on your own computer and makes no calls to outside services, so it has no restrictions of its own for Russia. The availability of the model that drives it is a separate matter.
How to disable. Remove the server from the MCP client configuration and restart the client: the model then loses access to the mouse and keyboard. If you started the server by hand in HTTP mode, stop the process.
FAQ
How do I connect computer-use-mcp to Claude Code?
Add a server to the MCP configuration with the command npx and the arguments -y computer-use-mcp. The author points to the install-mcp config generator, which builds the right fragment for Claude Code, Claude Desktop, Cursor, Cline and VS Code.
How does computer-use-mcp differ from Anthropic's computer use tool?
The tool scheme is nearly the same as in Anthropic's guide. The difference is that the server starts locally with one command and nudges the model to rely more on keyboard shortcuts.
Which actions does the model get?
Pressing a key, typing text, moving the cursor, single, double, right and middle click, dragging, scrolling, reading the cursor position and taking a screenshot. All of it is one computer tool with an action parameter.
Is it safe to give a model control of the computer?
The author calls it probably a bad idea and compares it to a hyperactive toddler at your computer. Models make mistakes and are vulnerable to injections, so watch the work and, where possible, run the server in a separate account.
How do I make clicks more accurate?
Pick a small resolution such as 720p, use a model that is strong at these tasks and install the Rango extension for keyboard navigation on websites. On screenshots the model navigates by the red crosshair that marks the cursor position.
Discussion
No questions or reviews yet.
Related
MCP serversA Rust MCP server that lets an agent control the desktop and the Chrome browser on your own computer
An MCP server and Chrome extension that let the agent drive your already open browser with your logged-in sessions, all locally
MCP gives an agent a Playwright browser where a person can take over the same session and save an authentication profile
A browser extension and local proxy that bring MCP tools into web chat UIs like ChatGPT, Perplexity and Grok
MCP Server Playwright by Automata Labs
MCP Server Playwright
Playwright-based MCP server for browsing sites, clicking, filling forms, taking screenshots and running JavaScript in a browser
An MCP server that lets an agent drive a real browser: open pages, click and fill forms through Playwright
README in short
The README is short: a safety warning, a link to the config generator and tips on models, screen resolution and the Rango extension. The how-it-works section says the tool nearly repeats Anthropic's official computer use guide and that the server talks to the system through the nut.js library.