mac-computer-use

Install

npx skillfoxx add skills/mac-computer-use

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a claude-code -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a cursor -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a github-copilot -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a codex -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a gemini-cli -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a cline -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a roo -y

npx skills add to3akarin/mac-computer-use --skill mac-computer-use -a opencode -y

git clone https://github.com/To3akaRin/mac-computer-use.git
cd mac-computer-use
bash scripts/build.sh

This is third-party code. Review the repository files before installing.

What it does

The skill gives the agent tools to work with real macOS applications. The agent lists windows and processes, reads the accessibility tree, takes screenshots and performs clicks, typing and shortcuts. For embedded Chromium pages there is control over the CDP protocol: reading structure, inserting text and running read-only assertions. The loop is built on observation and verification: inspect the target, pick a channel, run one action, then read the result and confirm it. Write commands support a dry run, and stale window references are rejected.

Who it is for. For engineers and testers who need macOS desktop app automation with result verification.

Example request

Find this app's window, type text into the right field and read it back to verify

Limitations

It runs only on macOS 14 and newer, needs Swift 6 and accessibility and screen recording permissions; CDP also needs Node.js 22.4+. The first native call compiles the binary, which is an expected wait. A test app and an isolated browser are verified; not every third-party program is guaranteed to work.

How to disable. Remove the mac-computer-use skill folder from your agent's skills directory. Optionally revoke the host app's accessibility and screen recording permissions in system settings.

Security check

• Runs local commands and controls apps: it clicks, types and changes program state
• Requires the host's accessibility and screen recording permissions

README in short

The README presents mac-computer-use as a skill under the open Agent Skills format that gives an agent direct control of a Mac. It lists observing windows and processes, reading and changing accessibility elements, native clicks and typing, plus embedded Chromium control over CDP. It stresses execution boundaries: dry-run write commands, explicit coordinate units, orderly foreground use and returning an unknown status when a result cannot be confirmed. There is a shared entry via scripts/run.sh for different clients and a test suite. MIT licensed.

SKILL.md

---
name: mac-computer-use
description: 操作 macOS 桌面应用，探测窗口和自动化接口、截图、读取或修改辅助功能元素、执行鼠标键盘动作，以及通过 CDP 操作内嵌 Chromium 页面。适用于桌面应用自动化与界面验收；普通网页任务优先使用已有浏览器工具。
license: MIT
compatibility: 需要 macOS 14+、Swift 6+、可写技能构建目录，以及读取文件和执行本机命令的能力；CDP 需要 Node.js 22.4+。桌面控制需要宿主的辅助功能和屏幕录制权限。
metadata:
  author: To3akaRin
  version: "0.2.0"
---

# Mac Computer Use

本技能使用本目录中的独立工具，不需要专属 MCP 插件。所有相对路径均以本文件所在目录为基准；Agent 的当前目录可能是其他项目，执行时先解析技能根目录并使用绝对路径。

## 通用入口

本目录遵循开放的 Agent Skills 格式。使用 runtime 提供的文件读取、本机命令执行和图片查看能力；agents/openai.yaml 是可选界面元数据，不参与核心运行。

```bash
sh "$SKILL_DIR/scripts/run.sh" native doctor
sh "$SKILL_DIR/scripts/run.sh" native windows
sh "$SKILL_DIR/scripts/run.sh" cdp targets --endpoint http://127.0.0.1:9222
```

## 开始前

1. 通过统一入口运行 native doctor 检查平台与权限。
2. 明确目标应用、窗口与用户要得到的结果。
3. 先 probe 和观察。优先应用已有 CLI、脚本字典或接口，其次 AX，再考虑鼠标键盘。

## 操作循环

观察目标 选定元素或坐标 写操作预演 执行一次 读回目标状态 判断完成或重新观察。

Related