SkillFoxx

webclaw CLI

Local-first Rust tool for web extraction: scrape, crawl and structured data for LLMs, with a CLI, REST API and MCP server

  • 2,440 ★ on the repository
  • Last commit
  • AGPL-3.0
CLI

Extended access

Why this level

  • Fetches content from the network
  • Runs code on the local machine
All reasons and checks

0xmassi/webclawAuthor:0xMassi

Install

Manual install

{
  "mcpServers": {
    "webclaw": {
      "command": "npx",
      "args": ["-y", "@webclaw/mcp"]
    }
  }
}

MCP hookup with no source build.

Access and checks

How we review

Extended accessThe tool runs code or makes network calls and only reads external data.

  • Fetches content from the network
  • Runs code on the local machine

This is third-party code. Review the repository files before installing.

What it does

webclaw pulls page content and turns it into markdown or structured data for language models. It runs locally in Rust, so requests do not go through a third-party service. There are three ways to use it: a terminal command, a REST API and an MCP server for agents. The MCP mode connects via npx with no source build, and create-webclaw detects installed AI tools and writes their configs for you.

Who it is for. For developers and analysts who need local web content collection for agents and pipelines.

Good fit when

  • You need page content as markdown for an agent
  • You need local scraping without an external service
  • You want to add web extraction to an MCP client

Not a fit when

  • You want a managed cloud scraping service with support
  • Target sites require bypassing anti-bot protection

Example request

Scrape these competitor pages into markdown and compare the prices

Limitations

Building from source needs Rust and system libraries, and a first build takes several minutes. The prebuilt ZIP and the npx launcher are the easiest path. The AGPL-3.0 license imposes conditions on derived products.

How to disable. Remove the webclaw entry from your MCP config, and remove the installed CLI the way you installed it (brew, cargo or deleting the ZIP).

FAQ

Do I need Rust for the MCP mode?

No, the MCP server runs via npx with no build. Rust is only needed to build the CLI from source.

Does data leave to a third-party server?

No, extraction runs locally on your machine.

Discussion

No questions or reviews yet.

CLI tools
Official

A Rust server that turns pages into Markdown or JSON: search, scrape, site map, crawl and PDF through MCP, a CLI and a Firecrawl-compatible API

MCP serverExtended access1.3K

A plugin that crawls sites and turns pages into Markdown with metadata: seven skills and an MCP server with nine tools

PluginBroad access196
Editors’ pick

The Firecrawl command-line client with agent skills: web search, scraping, crawling, URL mapping and page interaction

CLIExtended access684

servo-fetch renders pages with the Servo engine and gives an agent Markdown, JSON or a screenshot with no Chromium.

CLIExtended access164

An MCP server and CLI that crawls sites: a small model picks links and passages so the agent reads only the answer.

MCP serverExtended access89

MCP server with DuckDuckGo search and page text extraction: several URLs can be passed in one call

MCP serverExtended access151

README in short

The README describes a fast local Rust tool for web content extraction for LLMs. Install via npx create-webclaw, brew, a prebuilt ZIP, cargo or Docker. It returns markdown and structured data and works as a CLI, REST API and MCP server. For a zero-install agent hookup you point your MCP config at the @webclaw/mcp npx launcher. AGPL-3.0 licensed.

README badge

Report a problem