OpenGUI skill
Skills and a CLI that let Claude Code, Codex and OpenCode operate an Android phone: open apps, tap, type and test workflows
- 1,803 ★ on the repository
- Last commit
Broad access
Why this level
- The agent taps, types and opens apps on a real phone that holds personal messages, banking and other accounts
- Screen screenshots go to the chosen model, and the client gets the AccessibilityService and USB debugging
- The plugin installer runs through curl piped into bash and prepares an environment on your machine
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/openguiDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The agent taps, types and opens apps on a real phone that holds personal messages, banking and other accounts
- Screen screenshots go to the chosen model, and the client gets the AccessibilityService and USB debugging
- The plugin installer runs through curl piped into bash and prepares an environment on your machine
This is third-party code. Review the repository files before installing.
What it does
OpenGUI lets an AI agent see a real Android phone's screen through screenshots and operate app interfaces. The agent opens apps, taps, scrolls, types text, goes back and works through multi-step flows. Tasks are written in plain language: test a feature in an app and log issues with steps and screenshots, read the screen content, fill a form or prepare a social media draft. The repository ships two skills for agents. `open-gui-bootstrap` installs and configures the backend and the Android client when asked something like "run OpenGUI", and only asks the person to do things on the phone. `open-gui-remote-control` drives the phone through the repository's CLI: `pnpm opengui -- devices --json` lists connected devices, `do` starts a task and returns an `executionId`, `status` checks progress and `cancel` stops it. The documentation forbids raw `adb shell input`. There are also ready plugins for the WorkBuddy and DeepSeek Harness hosts: the phone connects over USB and no full backend is needed. The model doing the work must accept images and call tools.
Who it is for. For mobile developers and testers who want Claude Code or Codex to run scenarios on a real Android phone.
Good fit when
- You need to run a scenario in a mobile app on a real phone and get a report with reproduction steps and screenshots
- An agent should open the phone settings, read a value and return to the home screen
- You need to prepare a draft post in a social media app and stop before sending
- You want to dispatch phone tasks from Claude Code, Codex or OpenCode with `pnpm opengui -- do`
- You use WorkBuddy or DeepSeek Harness and want to add phone control through a plugin
Not a fit when
- You need an OSI-approved open license or commercial use: the source is under BUSL 1.1, which forbids production and commercial use without a separate license
- You have no Android phone with USB debugging on: ADB and device authorization are required
- You need an iPhone: the project is described for Android only
- You cannot send screen screenshots to a model: tasks run through a vision-capable model
Example request
Through OpenGUI open Settings on the phone, find the Android version and return to the home screen. Change nothingLimitations
- BUSL 1.1 license: you may copy, modify and use the source for non-production purposes. Production and commercial use, hosted services and integration into commercial products need a separate Core-Mate license. The change date to Apache 2.0 is 30 April 2030; until then it is not OSI-approved open source.
- The full stack needs an Android 11 (API 30) or newer client, USB debugging, the AccessibilityService, overlay permission and a battery optimization exemption.
- The WorkBuddy plugin runs on macOS with WorkBuddy 5.5.3 or later and is in public pre-release testing. The DeepSeek Harness plugin needs Node.js 22.19 or 24.
- Execution needs a model with image input and tool calling; the author advises evaluating quality, latency and cost on your own tasks.
- WorkBuddy SMS sign-in uses the CoreMate account service, and task screenshots go to the chosen model.
- Documentation and some guides are written in Chinese.
How to disable. Stop the backend and client, then revoke USB debugging and the AccessibilityService permissions on the phone. For the WorkBuddy and DeepSeek Harness plugins, disable or remove the plugin in the host settings. The DSH installer for macOS ships an `uninstall-macos.sh` script.
FAQ
How do I run OpenGUI from Claude Code?
Open Claude Code, Codex or OpenCode in the root of the cloned repository and send: "Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions." The skill installs the backend and client and asks you only for phone-side steps.
How do I send a task to the phone from a command?
From the `server` folder run `pnpm opengui -- do "Observe the current Android screen, summarize what you see, and stop" --json`. The command returns an `executionId` at once, with which `status` shows progress and `cancel` stops the run.
Which permissions does the phone need?
For the full stack the client needs Android 11 (API 30) or newer, USB debugging, the AccessibilityService, overlay permission and a battery optimization exemption. The WorkBuddy and DeepSeek Harness plugins only need authorized USB debugging.
Can I use OpenGUI in a commercial project?
Only with a separate license. The BUSL 1.1 source allows non-production use, while production, commercial use, hosted services and integration into commercial products need Core-Mate's permission.
Is the full backend required?
No, if you use WorkBuddy or DeepSeek Harness: the plugin installs separately, the phone connects over USB and screenshots go to the host's current model. The backend is for self-hosting and for dispatching tasks through Feishu, Telegram, Discord or REST.
Discussion
No questions or reviews yet.
Related
SkillsSkill and CLI for controlling an Android phone via ADB and vision models: the agent opens apps, searches, orders and verifies results
An open phone agent model and framework: it reads the screen and taps through Android, HarmonyOS and iOS apps from a text command
Awesome Android Agent Skills
awesome-android-agent-skills
Skills for modern Android development for Copilot, Claude, Gemini and Cursor: Kotlin, Jetpack Compose, architecture, Hilt, Room
Official set of 30 skills from the Android team: Jetpack Compose, AGP 9 upgrade, R8, Perfetto profiling, Navigation 3, CameraX, Google Play policies
Lets Claude Code and Codex drive a real iPhone: read the screen, tap, type and scroll in any app
A headless Android debugger on JDWP: the agent sets breakpoints and reads variables, and every answer comes back as JSON
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| OpenGUI (this entry) | no | 1,803 | ||
| Mobile MCP | no | not required | 10,095 | an MCP server for driving devices, while OpenGUI offers skills and a CLI with tasks, statuses and cancellation |
| Android MCP | no | not required | 907 | an MCP server for Android, while OpenGUI installs its own backend and client and runs tasks asynchronously through an executionId |
README in short
The front page presents OpenGUI as a mobile GUI agent framework for Android and immediately offers two simple paths: a WorkBuddy plugin and a DeepSeek Harness plugin. A table of common commands follows, then step-by-step plugin installation, running the full stack through the bootstrap skill and the CLI for remote control. The page closes with requirements, limitations, the roadmap, a system diagram and the license terms.
README badge
Original SKILL.md
--- name: open-gui-remote-control description: Control an Android phone through OpenGUI from an AI coding agent such as Claude Code, Codex, or OpenCode. Use when an agent should list online devices, run a natural-language mobile task, check execution status, pause, resume, or cancel through the local OpenGUI backend and CLI. --- # OpenGUI Remote Control Use this skill when an AI coding agent such as Claude Code, Codex, or OpenCode needs to operate an Android phone through OpenGUI. The agent should not drive the phone with raw `adb shell input` commands. The supported path is: ```text Claude Code / Codex / OpenCode -> server CLI or REST API -> OpenGUI backend task/execution services -> standby dispatch -> Android client -> execution socket action loop ``` ## Trigger Guidance Start this skill when the user asks for any of these: - "Use OpenGUI to control my phone" - "Use OpenCode to control my phone through OpenGUI" - "Run this task on the Android device" - "Use the OpenGUI CLI" - "List OpenGUI devices" - "Check / pause / resume / cancel an OpenGUI execution" If the user asks to install or bootstrap OpenGUI from scratch, use `open-gui-bootstrap` first. After backend and Android client are running, return to this skill for task execution. ## Core Rules - First obtain or locate a runnable OpenGUI checkout. The CLI cannot work without the repository. - Use the repository CLI first: `cd server && pnpm opengui -- ...`. - Use `--json` whenever the result will be parsed by the coding agent. - Do not ask the user to run terminal commands that the agent can run. - Ask the user only for physical phone actions, Android permissions, or missing secrets. - Do not change Android socket event names or payloads. - Do not use IM commands for this workflow. - Do not bypass OpenGUI with coordinate-only `adb shell input` scripts.