SimAgentation plugin
A plugin and MCP server that give an agent annotations on the iOS Simulator screen: element, screen, search strings, two shots.
- 62 ★ on the repository
- Last commit
- Apache-2.0
Extended access
Why this level
- The tool uses private Xcode frameworks and controls the simulator: it boots, creates and resets devices.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add plugins/simagentationDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The tool uses private Xcode frameworks and controls the simulator: it boots, creates and resets devices.
This is third-party code. Review the repository files before installing.
What it does
SimAgentation brings the Agentation idea to iOS. A browser page shows a running simulator, the user freezes the screen, clicks an element or drags a box over an area and writes what should change. The agent gets the element itself, the screen, strings to search the source and two shots: the whole screen with a red box and a close-up of the box. The app needs no SDK: everything comes from the simulator accessibility tree so any app works, system ones included. An MCP server with seven tools hands annotations to the agent, and in Claude Code each annotation can land in a running session right after sending. All pending annotations can also be copied as text. For your own apps an optional SDK gives the exact file and line for each tagged view.
Who it is for. iOS developers who edit interfaces through an agent and are tired of describing in words which element on screen is wrong.
Good fit when
- A designer points at a button in the simulator and the agent must find it in code.
- Several elements must be marked at once with a single note.
- Notes from the screen should be pasted as text into any chat.
- An app with the SDK needs an exact file name and line for a view.
Not a fit when
- You test on a physical device or Android: the tool works only with the iOS simulator.
- There is no Apple Silicon Mac with Xcode 26.
- A project without private Xcode frameworks is required.
Example request
Open SimAgentation, take my annotations on the sign-in screen and find the screens and lines in the source that need changes.Limitations
An Apple Silicon Mac and Xcode 26 are needed, and Xcode 27.1 for the foldable iPhone Duo. The native side uses the private CoreSimulator, SimulatorKit and AccessibilityPlatformTranslation frameworks, so a new Xcode release may break the tool until it is updated. On first start the MCP server downloads a release of about 2 MB, checks its SHA-256 and keeps it in `~/Library/Caches/sim-agentation`. The page opens on local port 38470 and a simulator boots only after you press a button.
How to disable. Remove the plugin with `/plugin uninstall sim-agentation@sim-agentation` and the package with `brew uninstall sim-agentation`. Clear the cache in `~/Library/Caches/sim-agentation` by hand.
FAQ
How do I install it?
Run `brew install lcandy2/tap/sim-agentation` and `sim-agentation serve --open`. The agent plugin downloads the matching version itself.
Does the app need an SDK?
No. Everything comes from the simulator accessibility tree. The optional SDK gives exact file and line for views.
How do I install the plugin in Codex?
Run `codex plugin marketplace add https://github.com/lcandy2/sim-agentation.git`, then `codex plugin add sim-agentation@sim-agentation`.
How do I start annotating?
Press Shift+Command+D: the screen freezes, then pick an element or drag a box and write a note.
What does the agent get?
The element, the screen, strings to search the code and two shots: the screen with the box and a close-up.
Discussion
No questions or reviews yet.
Related
PluginsAn MCP server for the iOS Simulator: taps, swipes, text input, accessibility tree, screenshots, video and app control
Claude Code plugin for iOS: eight small MCP servers and eight skills to build, test and drive the simulator.
A CLI that shows an agent an iOS Simulator or Android emulator screen as a compact element list and taps elements by short alias.
An MCP server that takes plain-language commands and drives the iOS Simulator through idb: apps, taps, logs
A Callstack plugin that shows simulators and devices beside the agent chat and lets it control the mobile app.
Desktop app and CLI for Android emulators, iOS simulators and real phones with an MCP server for agents
README in short
The README covers what the tool does, features, running and building, annotation mode, connecting an agent through the plugin and skills, the optional SDK and settings.