PullMD: web pages and files as Markdown for agents
PullMD
A self-hosted Docker service that turns links, documents, YouTube and Reddit threads into clean Markdown, reachable as MCP, a Claude Code skill or an API
- 492 ★ on the repository
- Last commit
- AGPL-3.0
Extended access
Why this level
- The service fetches addresses the agent hands it and returns their content, and the text of a third-party page can carry instructions
- Optional modules take keys for images, audio, OCR and Reddit, and with authorization off the service is open to anyone on the network
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add mcp/pullmdDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The service fetches addresses the agent hands it and returns their content, and the text of a third-party page can carry instructions
- Optional modules take keys for images, audio, OCR and Reddit, and with authorization off the service is open to anyone on the network
This is third-party code. Review the repository files before installing.
What it does
PullMD takes a page address or a file and returns Markdown without navigation, banners and scripts. The source and metadata sit in a YAML header rather than in the text. For web pages the service tries a chain: Markdown served natively by Cloudflare, then Readability with Trafilatura, then headless Chromium for JavaScript pages. It parses Reddit and Hacker News threads together with their comment trees, PDF, Word, PowerPoint, Excel and EPUB, and YouTube transcripts with clickable timecodes and no API key. Images and audio go through any OpenAI-compatible or local model. The query parameter returns only the sections that match a question and cuts output by 70-95 percent of tokens. There are three ways to reach it from an agent. A remote MCP server at /mcp offers the read_url, get_share and list_recent tools. The Claude Code skill installs from the pullmd.zip archive, and a universal prompt fits any chat. All of it runs on your own server through docker compose.
Who it is for. For developers and teams that run their own server and want agents to read sites, documents and videos more cheaply in tokens than raw HTML.
Good fit when
- The built-in fetch tool fails on PDFs, JavaScript sites or Reddit threads where a bot check blocks a plain request
- The agent needs an answer from a long article that does not fit the context: the query parameter returns only the matching sections
- You need a YouTube transcript with timecodes for notes or a report
- You want a permanent /s/<id> link to a parsed page: the service refreshes a copy older than an hour and serves the last good snapshot if the source disappears
- You connect an agent from Claude Desktop or claude.ai: PullMD has OAuth 2.1 for them
Not a fit when
- You have no server or cannot run Docker: the service installs only on your own machine
- You need to get around restricted access or paywalls: PullMD reads publicly available pages and documents
- Disk space is tight: the optional Playwright container adds about 3.7 GB to the image cache, although it can be left out
- A maintenance-free cloud service would do: PullMD has to be deployed, updated and secured by you
Example request
Read this article through PullMD and answer only the question about rate limits without loading the whole pageMCP
- Transport
- http
- Authentication
- key is optional
Environment variables
PUBLIC_URL- Public address of the service, http://localhost:3000 by default
OAUTH_JWT_SECRETsecret- Secret for OAuth 2.1, needed for Claude Desktop and claude.ai
PULLMD_TRUST_PROXY- Turns on trust for a reverse proxy
PLAYWRIGHT_URL- Address of the optional Playwright container for JavaScript pages
Limitations
- You deploy and update the service yourself. The license is AGPL-3.0.
- All side containers (Trafilatura, Playwright, MarkItDown) are optional, and without them page and document parsing gets weaker, which the service reports in its metadata.
- Claude Desktop does not accept a Bearer header in its connect form, so it needs OAuth enabled through OAUTH_JWT_SECRET or a proxy with the token in the URL path.
- Behind a reverse proxy since version 3.13 the PULLMD_TRUST_PROXY variable is required.
- A YouTube response can come back as blocked or error. These are transient states and are not cached.
How to disable. Run claude mcp remove pullmd and delete ~/.claude/skills/pullmd if you installed the skill. The service itself stops with docker compose down in the folder holding docker-compose.yml, and its data lives in ./data.
FAQ
How do I install PullMD in Claude Code?
Start the service first: download docker-compose.yml and run docker compose up -d. Then connect MCP with claude mcp add --transport http pullmd https://your-host/mcp. Instead of MCP you can download pullmd.zip from your service and unpack it into ~/.claude/skills/.
Which tools does the PullMD MCP server provide?
Three: read_url reads an address, get_share opens a permanent link and list_recent shows recent conversions. With authorization on, list_recent lists only yours.
How many tokens does PullMD save?
The author measured three pages with the cl100k_base tokenizer: an encyclopedia article is 99,458 tokens through curl against 12,549 through PullMD, and a documentation page 48,611 against 6,591. With the query parameter the figures are 662 and 454.
Does PullMD read Reddit when a plain request is blocked?
Yes. Reddit threads are detected automatically, including redd.it short links. In the author's measurement plain curl hit a bot check while PullMD returned the thread with comments at 6,381 tokens.
Which license does PullMD use and where is the data stored?
AGPL-3.0. The cache and history live in the ./data directory that docker-compose mounts as a volume.
Discussion
No questions or reviews yet.
Related
MCP serversAn MCP server that opens pages in headless Chromium through Playwright and hands the agent the main text as Markdown, even from JavaScript-heavy sites
A skill that reads a link and hands the agent the full text as Markdown, picking a separate route for WeChat pages, Feishu docs, PDFs and arXiv papers
A Rust server that turns pages into Markdown or JSON: search, scrape, site map, crawl and PDF through MCP, a CLI and a Firecrawl-compatible API
An open-source crawler that turns websites into clean Markdown for AI agents: install from PyPI, run in Docker and connect to Claude Code over MCP
Web search via DuckDuckGo and page reading for the agent with no API keys, with built-in rate limiting
MCP server for Reddit: an agent reads hot threads from a subreddit and analyzes posts with their comments
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| PullMD: web pages and files as Markdown for agents (this entry) | no | key is optional | 492 | |
| MarkItDown | yes | not required | 191,532 | Microsoft's utility for converting files to Markdown, without the web fetch chain or a caching server |
| Jina AI MCP | yes | key is optional | 892 | Jina's cloud service, while PullMD runs on your own server |
| webclaw | no | not required | 2,386 | a local Rust tool for extracting data from pages and crawling sites |
README in short
The Why section explains the task: raw HTML spends context on navigation, banners and scripts, and built-in fetch tools stumble on JavaScript, PDFs and discussions. Next come a token savings table, the list of formats, a docker compose quick start and the three ways to connect agents. Separate sections cover authorization modes, MCP client compatibility, OAuth for claude.ai and Claude Desktop, and reverse proxy setup. API and parameter references close the document.