agent-vision-toolkit skill
A set of CLI tools and a skill that give a text-only agent sight: image analysis, long-screenshot OCR and UI restoration
- 1,260 ★ on the repository
- Last commit
- MIT
Extended access
Why this level
- Sends images to an external multimodal API
- Runs CLIs and can automate a GUI
Install
- Works in
Manual install
npx skills add Anionex/agent-vision-toolkit --skill vision-skills -gAfter install, set VISION_API_KEY, VISION_BASE_URL and VISION_MODEL.
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- Sends images to an external multimodal API
- Runs CLIs and can automate a GUI
This is third-party code. Review the repository files before installing.
What it does
agent-vision-toolkit adds sight to agents on text-only models such as DeepSeek that lack multimodality. It consists of several CLIs and a skill that teaches the agent when to use each: image Q&A, long-screenshot OCR, frontend UI restoration and GUI automation. Any agent that can call shell commands can use them. There is also a transparent local proxy and single-file plugins so pasted images and built-in image tools work without extra setup.
Who it is for. For people running an agent on a text-only model who want to give it image handling.
Good fit when
- The agent's model cannot see images but you need to work with them
- You need OCR of a long screenshot or image analysis
- You want to restore markup from a UI screenshot
Not a fit when
- The agent's model is already multimodal
- You have no access to an external multimodal API
Example request
Read the text on this long screenshot and describe what it showsLimitations
It needs a multimodal API compatible with OpenAI Chat Completions, OpenAI Responses or Anthropic Messages: its base URL, key and model. Availability depends on the chosen provider.
How to disable. Remove the bin directory from PATH and delete the installed skill or plugins from the agent directory.
FAQ
What do I need to prepare?
A multimodal API compatible with OpenAI or Anthropic: a base URL, key and model name.
Which agents does it support?
Codex, Claude Code, Pi, Oh My Pi and OpenCode.
Discussion
No questions or reviews yet.
Related
SkillsAn open coding agent for the terminal, browser and desktop that works with models from OpenAI and Claude to Ollama
A skill that gives text-only models sight: an image by path or from a paste becomes structured JSON with transcribed text, layout and semantics
Agents Best Practices
agents-best-practices
A provider-neutral skill for designing agent systems: loop architecture, tools, permissions, memory and observability
A CLI installer and package manager for coding agent skills: one command installs skills into many runtimes and helps build your own
A CLI to run several coding agents: one config, a session console, local memory and a set of engineering skills
A multi-agent harness that verifies results by artifacts: stop hooks, an independent judge and an event log for Claude Code, Codex, Cursor and more
README in short
The README explains that an agent's vision can move out of the model and into the harness: a set of CLIs and a skill teach a text-only agent to analyze images, OCR long screenshots, restore UI and automate GUIs. A local proxy and single-file plugins install optionally for seamless image handling. An external multimodal API is required. MIT licensed.
README badge
Original SKILL.md
--- name: vision-skills description: >- Local vision CLIs: glance (describe/ask/OCR an image), ground (locate a target, pixel box), detect (element inventory), trace (image to SVG geometry), crop (cut a pixel box to a file), and scripts/html_shot.py (HTML file to image). Use for any task involving an image — questions, text, splitting and transcribing long screenshots or chat histories, locating elements, comparing, rebuilding as HTML/SVG, digitizing a sketch or diagram, reading values off a chart, operating a GUI from screenshots — and to re-check an image yourself when a description you were given lacks a detail. --- # vision-skills Five local CLIs that give a text-only agent eyes. They read one shared vision config (`VISION_API_KEY` / `VISION_BASE_URL` / `VISION_MODEL` / `LANG`), plus the optional Python-client settings `VISION_API_PROTOCOL`, `VISION_REASONING_EFFORT`, and `VISION_USER_AGENT` — no extra credentials. Pick the tool by the question you are answering: | Question | Tool | |---|---| | "What does this image show / say?" | `glance` | | "Where is X?" — a thing you can name | `ground` | | "Where are all the Xs?" — every instance of a kind | `detect` | | "What is its exact shape, size, offset?" | `trace` | | "Cut this box out as its own image file" | `crop` | | "OCR this long screenshot / scrolling page / chat history" | `scripts/long_screenshot_ocr.py` | | "Extract the icon/logo foreground as transparent PNG — manual region or auto (cropped+scaled screenshots)" | `scripts/extract_fg.py` | | "Turn this HTML file into a viewport or full-page screenshot" | `scripts/html_shot.py` | | "Which colours dominate a region, and which palette value fits it?" | `scripts/dominant_colors.py` | | A relation none of them return — a gap, a distance between two located things | code over the pixels (Pillow) | `glance` answers what something is; `ground` and `detect` answer where. You give `ground` a description of a particular thing; you give `detect` a kind and it enumerates the instances. Both give real coordinates, but they are not pixel-exact: the box arrives on a 0-1000 grid and is scaled to your image, so the last pixel or few are not reliable. That is accurate enough to crop with, to click, to compare positions against. When a number has to be exact, `trace` derives it from the actual pixels — offsets, sizes, shapes. ## Use the provided tools before hand-rolled pixels Everything this toolkit ships a tool for, call the tool — do not rewrite