HasData Agent Skills: scraping and SERP API
HasData Agent Skills
Two HasData skills teach an agent to call the Google SERP, Amazon, Zillow and Maps APIs without guessing endpoints
- 14 ★ on the repository
- Last commit
Extended access
Why this level
- Requests go to an external service and spend the account quota, and the API key sits in an environment variable
- Scraping other sites can break their terms and personal data law, especially when collecting people's contacts
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/hasdata-agent-skillsDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- Requests go to an external service and spend the account quota, and the API key sits in an environment variable
- Scraping other sites can break their terms and personal data law, especially when collecting people's contacts
This is third-party code. Review the repository files before installing.
What it does
HasData is a cloud platform for collecting public web data: search results, ready parsers for platforms and asynchronous jobs. The repository holds two skills that give an agent the right endpoints, parameters and response shapes. The hasdata skill wires API calls into Python, TypeScript and Go code. It covers `POST /scrape/web`, parsers for Google (results, AI Mode, Maps, News, Trends), Bing, Amazon, Shopify, Zillow, Redfin, Airbnb, Booking, Yelp, Indeed, Glassdoor, Instagram and YouTube, and the async job cycle: submit, poll, fetch results and webhooks. The references folder has code samples with retries and backoff. The hasdata-cli skill works through the hasdata console utility. Commands like `hasdata <subcommand> --flag value` make one-off requests whose output goes to jq, scripts and CI. The selection rule in SKILL.md: use a ready parser for a known platform, generic scraping for an arbitrary URL, and keep async jobs for bulk exports.
Who it is for. Developers building a service on search, marketplace and map data, and analysts who need quick exports from a terminal.
Good fit when
- A service needs Google results for a query and the code should receive ready JSON
- A list of cafes with sites and phones is collected from Google Maps for a lead base
- Prices and reviews from Amazon or Shopify go into a table for competitor analysis
- The documentation of a large site is crawled with an async crawler to get Markdown for a document corpus
- A bash script needs a one-off API request without writing a client
Not a fit when
- The data sits behind a login or belongs to your own account: the platform is built for public pages
- The budget is zero: every request spends the HasData account quota
- You want an MCP server: the repository has only skills and API guidance
Example request
Use HasData to fetch the first ten Google results for "buy a coffee machine" and save the titles with links to a CSV.Limitations
You need an account at app.hasdata.com and an API key in HASDATA_API_KEY. Requests spend quota: code 403 means it is exhausted, 429 means the concurrent request cap was hit.
The hasdata-cli skill works only with the hasdata command installed and configured. The skills give the agent the API schema, but an external service collects the data, and respecting source sites' rules stays with the user. The generic npx skills add command installs both skills at once.
How to disable. Delete the hasdata and hasdata-cli folders from the agent's skills directory. For the Claude Code plugin remove it with /plugin, and revoke the API key in the HasData dashboard.
FAQ
How does hasdata differ from hasdata-cli?
hasdata wires API calls into an application in Python, TypeScript or Go. hasdata-cli is for one-off requests from a terminal through the hasdata utility, with output into jq and scripts. Pick the first to build a service and the second for bash.
How do I install the skills in Claude Code?
Run /plugin marketplace add https://github.com/HasData/agent-skills, then /plugin install hasdata@hasdata-agent-skills and /plugin install hasdata-cli@hasdata-agent-skills. One of the last two is enough if you need a single skill.
Which response counts as successful?
Data is valid only when requestMetadata.status is ok. HTTP 200 alone guarantees nothing. Errors: 401 invalid key, 403 quota exhausted, 429 too many concurrent requests, 500 retry later.
Do the skills fit Cursor and Codex?
Yes, Claude Code, Cursor, Codex, Gemini CLI and any tool that reads SKILL.md fit. For Cursor and Windsurf the skill files are cloned into the project or a shared folder and the agent is pointed at the skills directory.
Where do I get a HasData API key?
In the dashboard at app.hasdata.com, under Dashboard and then API. Put the key in the HASDATA_API_KEY variable; for the hasdata-cli skill, configure the hasdata utility with hasdata configure.
Discussion
No questions or reviews yet.
Related
SkillsA skill for collecting business data from Google Maps: the agent plans queries, runs a local scraper and helps with the results
Official Exa skills connect an agent to the Exa API: semantic search, page extraction, answers, company research and lead lists
A web search skill with no MCP server: four command-line clients search the web and 16 vertical domains, in batches and with page extraction
The official Bright Data plugin: 21 skills for Google search, page scraping, ready scrapers, proxies and the Bright Data MCP server
Web Access
web-access
A web access skill that picks between search, page fetch and driving your own browser over CDP
Three skills and four utilities that let an agent search the web, read pages, pull video subtitles and process media
Comparison with similar tools
Cards that share the main search query, and how they differ.
| Tool | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| HasData Agent Skills: scraping and SERP API (this card) | yes | 14 | ||
| Bright Data Skills | yes | 271 | skills around Bright Data Web Unlocker, proxies and scrapers, while this covers the HasData API | |
| Apify Agent Skills | no | 2,497 | skills for the Apify actor platform, while this has ready HasData parsers and jobs | |
| SerpApi MCP Server | yes | API key | 184 | an MCP server for search results only, while this adds platform parsers and a crawler |
README in short
Each SKILL.md is self-contained, and the details live in references folders: web scraping, search, e-commerce, real estate, travel, local business, jobs, YouTube and async jobs. For Codex and Gemini CLI the README suggests reading the SKILL.md files directly. A HasData account and API key are required.
README badge
Original SKILL.md
---
name: hasdata
description: Use HasData to scrape any public web page, run real-time Google/Bing/Google-AI-Mode search queries, pull structured data from e-commerce, real-estate, lodging, jobs, maps, travel, video, and social platforms, or run async bulk-scraping and crawling jobs without managing proxies, browsers, or captchas. Reach for this skill when the user mentions web scraping, SERP/search results, Google Maps/Trends/Flights/Images, YouTube videos/transcripts/channels, Amazon, Zillow, Redfin, Airbnb, Booking.com hotels, Yelp, Indeed, Glassdoor, Instagram, Shopify, scraper jobs, website crawling, RAG/LLM data ingestion, lead/contact enrichment, or HasData itself.
---
# HasData
Cloud platform for extracting public web data. One API key, three execution modes. All endpoints sit under `https://api.hasdata.com` and authenticate with `x-api-key`.
```bash
curl -G 'https://api.hasdata.com/scrape/google/serp' \
--data-urlencode 'q=coffee' \
-H 'x-api-key: <your-api-key>'
```
`401` invalid key, `403` quota exhausted, `429` concurrency cap, `500` server error (retry).
## Three execution modes
| Mode | Latency | When | Endpoint |
|---|---|---|---|
| **Web Scraping API** | seconds | Arbitrary URL — JS rendering, CSS/AI extraction, screenshots | `POST /scrape/web` |
| **Scraper APIs** (sync) | seconds | Pre-parsed JSON for known platforms (Google, Amazon, Zillow, …) | `GET /scrape/<vertical>/<resource>` |
| **Scraper Jobs** (async) | minutes–hours | Bulk extraction, recursive crawling, webhook fan-out | `POST /scrapers/<slug>/jobs` |
**Decision rule.** Default to a **Scraper API** when one exists for the platform (pre-parsed JSON, no selector maintenance). Use **Web Scraping** for arbitrary URLs not covered by an API. Reach for a **Scraper Job** only when no API equivalent exists — `crawler`, `contacts`, `sec-edgar`, `amazon-bestsellers`, `amazon-product-reviews` — *or* when async fan-out + webhooks save engineering time over a paginated client loop.
## Always-true response shape
```json
{ "requestMetadata": { "id": "…", "status": "ok", "url": "…" }, "...": "endpoint-specific" }
```
Treat data as valid only if `requestMetadata.status === "ok"`. HTTP 200 alone isn't enough.
## High-leverage patterns
- **SERP-first enrichment.** Google SERP is a free-form data lake. `q="<Person> <Company> linkedin"` → `organicResults[0].title` + `.snippet` already carries role + location, no profile-page scrape needed. Same recipe with `crunchbase`, `wikipedia`, `github`, or quoted literals (`"jane@x.com"`, `"+1 555 …"`) for reverse lookup.
- **AI Mode + verify.** `/scrape/google/ai-mode` for the answer + references → `/scrape/web` (markdown) on each reference URL → cited RAG context, no vector DB.
- **Maps → leads.** `/scrape/google-maps/search` returns websites + phones; fan out to `/scrape/web` with `extractEmails: true` for full lead rows.
- **Crawler → corpus.** `crawler` Scraper Job with `outputFormat: ["markdown"]` + `includePaths: "/docs/.+"` produces an LLM-ready corpus in one submission.
- **Pre-extracted via SERP rich snippets.** `knowledgeGraph`, `localResults`, `inlineShoppingResults`, `relatedQuestions` carry pre-parsed facts that bypass anti-bot. Always check them before scraping.
## When to call from code (the wiring)