crawlie plugin
A plugin with an MCP server and skills: Claude Code crawls a site, finds broken links and SEO and GEO errors and suggests fixes.
- 117 ★ on the repository
- Last commit
Extended access
Why this level
- The crawler sends many requests to the audited site and without limits can load someone else's server.
- The MCP server starts through `npx`, so it downloads and runs an npm package on your machine.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add plugins/crawlieDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The crawler sends many requests to the audited site and without limits can load someone else's server.
- The MCP server starts through `npx`, so it downloads and runs an npm package on your machine.
This is third-party code. Review the repository files before installing.
What it does
crawlie is a crawler for technical SEO and GEO, meaning preparing pages to be cited by AI search. It runs locally, crawls a site respecting `robots.txt`, seeds from `sitemap.xml` and checks 40+ rules: broken links, redirects, metadata, markup, accessibility. Every finding comes with an explanation of why it hurts and how to fix it. The Claude Code plugin installs an MCP server and a set of playbook skills at once: `seo-site-audit`, `geo-ai-readiness`, `pre-launch-seo-check` and `fix-broken-links`. The agent crawls a site with `crawl_site`, checks single pages with `audit_url` and explains rules with `explain_issue`. The report comes out as JSON, CSV, HTML or terminal text. For CI there is a `--fail-on` flag that makes the command fail when problems of the chosen severity are found.
Who it is for. SEO specialists and landing page developers who want to check a site before launch right from Claude Code, without a paid crawler.
Good fit when
- A landing page is about to go to production, and 404s, redirect chains and empty title tags have to be caught first.
- An agent built the site, and the question is whether it is ready to be cited by AI search.
- A CI pipeline blocks a release when error-level issues appear on the site.
- A site report goes to a client as a single HTML file.
Not a fit when
- Rankings and keyword data are needed: the crawler checks the site itself only.
- The site sits behind a login: the crawl runs as an anonymous visitor.
- You want to audit someone else's site without the owner's consent: by default the crawler sends 16 parallel requests, which is real load.
Example request
Crawl example.com and name the five fixes that would lift the GEO score most, with the exact change for each.Limitations
The default cap is 500 pages and 16 parallel requests, changed with the `--max-pages` and `--concurrency` flags. The crawler respects `robots.txt`, but the check can be turned off with `--no-robots`, so point it only at your own sites. Without an installed crawlie the skills run it through `npx`, which needs Node.js and network access. The cloud MCP at `https://crawlie.app/mcp` requires an API key and spends the plan's quota. The desktop app ships for macOS only.
How to disable. Run `claude plugin uninstall crawlie@spronta`. A standalone install is removed with `claude mcp remove crawlie` and `npm uninstall -g crawlie`.
FAQ
How do I install crawlie in Claude Code?
Run `claude plugin marketplace add spronta/crawlie`, then `claude plugin install crawlie@spronta`. The plugin starts the MCP server through `npx` itself, so no pre-install is needed.
How does the GEO score differ from a regular SEO score?
Every crawl returns three separate scores: Health for technical SEO, GEO for AI-search readiness and Accessibility for WCAG. One kind of problem never hides another.
Can crawlie run in CI?
Yes. `crawlie crawl https://example.com --fail-on error` returns a non-zero code on errors, which can stop a release.
Does it work without Claude Code?
Yes. `npm i -g crawlie` is enough, and `crawlie-mcp` connects to any MCP client: Cursor, Cline or Claude Desktop.
How many pages does crawlie crawl at once?
Up to 500 pages by default, and the `--max-pages` flag changes the cap. `--max-depth` limits depth, and `--include` and `--exclude` scope the crawl.
Discussion
No questions or reviews yet.
Related
PluginsA Claude Code plugin for SEO audits: technical SEO, E-E-A-T, schema, local and AI search via parallel subagents
SEO audit plugin with Yandex-specific checks
claude-seo-pro
A Claude Code plugin with 15 skills and 10 parallel subagents: a full SEO audit including checks for Yandex Turbo Pages, Metrika and robots.txt
All-In-One Free SEO Tool
All-In-One-Free-SEO-Tool
A self-hosted SEO suite with an MCP server and five skills for audits, keywords and site fixes in Claude Code and Cursor.
An open CLI that audits a site against more than 300 rules and hands the agent fixes for SEO, performance and security issues
A Claude Code plugin for drafting SEO pages from search results and competitor material, with an outline, FAQ and schema
A Claude Code skill set for auditing sites for AI search: citability, AI crawler access, llms.txt, schema and reports
Comparison with similar tools
Cards that share the main search query, and how they differ.
| Tool | Official | Repository stars | Main difference |
|---|---|---|---|
| crawlie (this card) | yes | 117 | |
| SEOmator SEO Audit | no | 663 | a CLI and skill for site SEO audits; crawlie ships as a plugin with an MCP server and three separate scores |
| GEO-SEO Claude | no | 11,529 | a skill set for AI-search audits without its own crawler; crawlie brings a site crawl and an MCP |
README in short
The README walks through the npm install, the CLI commands with flags, connecting the MCP server and the cloud variant, the plugin and the skills. It continues with use cases, the list of checks and a comparison with other crawlers.