Tapilot plugin
A plugin for Codex and Claude Code: the agent views an Android phone's screen, taps and types over Wi-Fi, with no ADB or USB.
- 137 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- The agent drives a real phone and all its apps through the accessibility service, and by default acts without confirmation.
- Screen captures and task text are sent to the vision model provider.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add plugins/tapilotDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The agent drives a real phone and all its apps through the accessibility service, and by default acts without confirmation.
- Screen captures and task text are sent to the vision model provider.
This is third-party code. Review the repository files before installing.
What it does
Tapilot is an Android app that carries out tasks on a phone from a plain-language description. It is built on Midscene and works from screenshots. The `plugins/tapilot` plugin connects an already paired phone to Codex, Claude Code or another local agent with a shell and image reading. The agent calls the `tapilot` CLI (`setup`, `pair`, `preview`, `devices`, `connect`), views screenshots and decides each action itself, while Midscene writes a report with screenshots and plans. The link runs over Wi-Fi with the phone's pinned TLS certificate, with no ADB, USB, public server or tunnel. The phone must be unlocked and the Tapilot accessibility service enabled in Android settings.
Who it is for. Mobile developers and testers who want to hand an agent a scenario on a real phone.
Good fit when
- A scenario must run on a real Android device with no ADB.
- An agent checks how a screen looks and behaves after app changes.
- A routine action across several apps is described in words.
- A run report with screenshots of every step is needed.
Not a fit when
- An iPhone or a non-arm64 phone is required.
- Banking or other protected screens are open on the phone.
- Screen captures must not be sent to an external model.
Example request
Open Calculator on the phone, compute 12 × 8 and send a screenshot of the result together with a step report.Limitations
An arm64 phone with Android 11 or newer, the beta app build and Node.js 22.19 or newer for the plugin are required. The project is experimental: the model can misread a screen, and protected or locked screens and custom text inputs are unsupported. The app defaults to automatic execution (YOLO) without confirmations, so choose a confirmation mode before the first task. Screenshots and task text go to the chosen vision model provider.
How to disable. Cut access with `tapilot disconnect --device-id <id>` and remove the phone entry on the computer with `tapilot forget --device-id <id>`. Turn off the Tapilot accessibility service in Android settings.
FAQ
Are ADB or USB needed?
No. The link runs over Wi-Fi through the paired app with a pinned TLS certificate, and ADB, USB, public server and tunnel are not used.
What if the computer's address changed?
The next observation reconnects by itself and tries the discovered addresses. If discovery is blocked, choose Scan to reconnect on the recovery screen and scan the code in the app.
How does `steps` mode differ from `ai`?
In `steps` mode, the default, the agent views screenshots and decides each action itself. `ai` mode needs separate OpenAI authorization.
Can I avoid giving the agent full access?
Yes. In the app choose confirmation for critical operations or for every action instead of automatic mode. The Take over button stops the task and returns control to you.
Which models work?
A supported vision model over an OpenAI-compatible HTTP or HTTPS API, or a ChatGPT (SIWC) connection where available. For a model on a local network give its address like `http://192.168.1.100:8000/v1`: `localhost` on the phone points to the phone itself.
Discussion
No questions or reviews yet.
Related
PluginsSkill and CLI for controlling an Android phone via ADB and vision models: the agent opens apps, searches, orders and verifies results
An app, CLI, MCP server and skill let Claude Code and Codex see and tap an unmodified iPhone, with no jailbreak and no cable.
A Callstack plugin that shows simulators and devices beside the agent chat and lets it control the mobile app.
Skills and a CLI that let Claude Code, Codex and OpenCode operate an Android phone: open apps, tap, type and test workflows
Awesome Android Agent Skills
awesome-android-agent-skills
Skills for modern Android development for Copilot, Claude, Gemini and Cursor: Kotlin, Jetpack Compose, architecture, Hilt, Room
MCP server for Android through ADB and scrcpy: an agent sees the screen, taps, types text and launches apps.
Comparison with similar tools
Cards that share the main search query, and how they differ.
| Tool | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| Tapilot (this card) | no | 137 | ||
| Midscene Skills | yes | 321 | Midscene skills drive browsers and devices from the agent; Tapilot installs an app on the phone itself and drives it over Wi-Fi without ADB | |
| Appium MCP | yes | not required | 517 | Appium runs sessions through a server and drivers; Tapilot works from screenshots through the accessibility service with no separate server |
README in short
The README describes the app, the action and screenshot-verification loop, APK installation, model and voice input setup. The plugin in `plugins/tapilot` has its own README, a `SKILL.md` and manifests for Codex and Claude.