Agent-S for OpenClaw skill
A skill that wires Agent-S into OpenClaw: the agent hands over a task in words and Agent-S clicks through desktop windows itself.
- 12,959 ★ on the repository
- Last commit
- Apache-2.0
Broad access
Why this level
- The agent controls mouse and keyboard and sees the whole screen, including personal windows.
- Screenshots go to the chosen model provider, and the local environment flag runs arbitrary code.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/agent-s-for-openclawDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The agent controls mouse and keyboard and sees the whole screen, including personal windows.
- Screenshots go to the chosen model provider, and the local environment flag runs arbitrary code.
This is third-party code. Review the repository files before installing.
What it does
The `integrations/openclaw` folder holds a skill and wrapper that let OpenClaw run Agent-S, an agent that operates graphical interfaces. OpenClaw reads `SKILL.md`, calls `agent_s_task` with a plain-language task, and Agent-S looks at the screen, plans steps and performs clicks and typing. Parameters are `task`, `max_steps` (15 by default) and `enable_reflection` (on by default). A task takes two to five minutes. The wrapper accepts Anthropic, OpenAI or Gemini keys plus a grounding model for screen elements, UI-TARS-1.5-7B by default.
Who it is for. OpenClaw users who need to automate desktop programs that offer no command line or API.
Good fit when
- A program has no CLI or API and the action needs the mouse.
- A scenario crosses several apps, such as Finder, TextEdit and a browser.
- An interface needs a check on a test machine: open, click, read the result.
- An OpenClaw agent must work out where to click from a screenshot.
Not a fit when
- The task has a CLI or API: those are faster and more reliable than clicks.
- Personal windows, mail or banking are open on the machine.
- A result is needed within seconds: a typical run takes minutes.
Example request
Use Agent-S to open Calculator, compute 12 times 15 and tell me the result.Limitations
Required are the `gui-agents` package, Tesseract for OCR, OpenClaw, a model key and a running grounding model (`AGENT_S_GROUND_URL`, `AGENT_S_GROUND_MODEL` and screen dimensions). The `--enable-local-env` flag allows arbitrary code execution, so leave it off. The agent sees the whole screen and controls input, so run it in a clean session or a virtual machine. The skill is written for OpenClaw and is not meant for Claude Code, Cursor or Codex. It is best shown on commands such as opening apps and typing text, so run long chains with a small `max_steps`.
How to disable. Delete the `~/.openclaw/workspace/skills/agent-s` directory. `pip uninstall gui-agents` removes the package; remove the keys from the environment.
FAQ
How do I install Agent-S for OpenClaw?
Copy the directory with `cp -r integrations/openclaw ~/.openclaw/workspace/skills/agent-s`, make scripts executable with `chmod +x` and check `which agent_s`.
How many steps does Agent-S take per run?
Up to 15 by default. `--max-steps` changes the limit and `--no-reflection` speeds up the run at the cost of accuracy.
Which keys are needed?
One of `ANTHROPIC_API_KEY`, `OPENAI_API_KEY` or `GEMINI_API_KEY`, plus a configured grounding model for screen elements.
What is the grounding model and why is it needed?
It is the model that locates screen elements for clicks. Recommended is UI-TARS-1.5-7B and requires the `AGENT_S_GROUND_URL` and `AGENT_S_GROUND_MODEL` variables.
Can it run without OpenClaw?
Yes, for testing: `./agent_s_wrapper.py "Open TextEdit and type Hello World" --max-steps 10 --json` runs a task from the command line.
Discussion
No questions or reviews yet.
Related
SkillsA skill with references teaches an agent to install, configure and repair the OpenClaw gateway, channels and security.
An agent fills fields and selects menu items in a local application, checking the screen before and after each action.
Thirteen skills turn an OpenClaw agent into a worker for paid jobs, while Guard caps its spending and stops loops
Turns one run of a task into a program an agent calls as a single tool, checking the record before and after the save
A language and runtime for skills an agent writes and a human approves: execution through allowlisted connectors and signatures
A skill and SQLite engine that watch thousands of sources on a schedule, detect changes by hash and keep an audit ledger.
README in short
The integration README lists requirements, environment variables and three install steps, then shows running from OpenClaw and the command line with a verification checklist.