SkillFoxx

agent-traffic skill

A Mintlify skill and package that tell AI agents, crawlers and bots from humans by headers and IP and catch impersonators.

  • 72 ★ on the repository
  • Last commit
  • MIT
Skill

Minimal access

Why this level

  • The skill and package only classify requests; IP range verification fetches operators published lists over the network.
All reasons and checks

mintlify/agent-trafficAuthor:Mintlify

Install

In your terminal, with SkillFoxx CLI

npx skillfoxx add skills/agent-traffic

Detects the agents on your machine, shows the access level and pins the version.

Other ways to install

Access and checks

How we review

Minimal accessThe tool runs nothing and makes no network calls.

  • The skill and package only classify requests; IP range verification fetches operators published lists over the network.

This is third-party code. Review the repository files before installing.

What it does

agent-traffic teaches an agent how to classify requests to a site. Its core is a tested dataset of user agents, headers and official verification sources of real agents and crawlers. The `classify` function takes headers and IP and returns a category (agent, bot and others), kind, name, operator and version, and every result explains itself. The package checks crawlers against operators published IP ranges and catches impersonators: a request with a GPTBot User-Agent outside OpenAI ranges gets the kind impersonator. The skill teaches an agent to apply the package and dataset in request handlers and stored logs. The package has zero dependencies and runs anywhere `fetch` does.

Who it is for. Site and documentation developers who want to know how much traffic comes from AI agents and serve them Markdown instead of HTML.

Good fit when

  • Logs must separate Claude Code, ChatGPT and Cursor from regular browsers and search bots.
  • A request claims to be GPTBot and must be checked against official IP ranges.
  • A Cloudflare Workers handler must serve agents Markdown and people the page.
  • You must decide who reads the docs: a developer, an agent or a training crawler.

Not a fit when

  • You need DDoS protection: classification does not block traffic itself.
  • The client IP is unavailable: crawler authenticity checks rely on the address.
  • The site blocks all bots by rule: the classifier only tells who came.

Example request

Use the access logs to find what share of documentation requests comes from AI agents and verify claimed GPTBot requests against IP ranges.

Limitations

The classifier identifies the kind of request and does not block it: the developer writes the reaction to the result. Verifying crawler authenticity needs the real client IP, for example the `cf-connecting-ip` header in Workers, and IP ranges are loaded on the first request and cached. The `headersComplete` mode adds extra fake browser signals. The dataset reflects Mintlify observations of documentation traffic and ages with time, so keep the package updated. The skill itself needs an agent that supports Agent Skills.

How to disable. Remove the skill from your agent skills folder and the package with `npm uninstall @mintlify/agent-traffic` if you added it to a project.

FAQ

How do I install the skill?

Run `npx skills add mintlify/agent-traffic` in Claude Code, Cursor, Codex or another agent supported by the skills CLI. You can also paste the skill URL straight into an agent.

What does `classify` return?

An object with category (agent or bot), kind, name, operator, version and the reasons. For a fake the kind becomes impersonator.

Where does the package run?

Anywhere `fetch` exists: Node, Bun, Deno, Cloudflare Workers and Vercel Edge. The package has no dependencies.

How do I catch a fake crawler?

Pass `ip` with the headers and load ranges through `loadIpRanges()`. A request with a GPTBot User-Agent outside OpenAI ranges gets the impersonator kind.

Where does the data come from?

Mintlify studied the traffic of its documentation sites, tested existing open bot lists and kept the signals that really separate requests.

Discussion

No questions or reviews yet.

Skills

Eleven SEO skills for Claude Code, Cursor and Codex: a daily autopilot, title fixes and post refreshes

SkillBroad access1.2K

A 25-point guide plus an auditor skill: Claude Code checks a site for AI-agent readiness and ranks the gaps it finds.

SkillExtended access106

Skill and CLI for SEO audits and website builds: a scroll story, GPT images, metadata checks and deployment to free hosting

SkillExtended access354

A skill set for SEO and indie development: keyword picking, launching a site on Cloudflare, backlinks, dashboard data and growth.

SkillExtended access304

SEO and AI-search optimization skill: a 100-point site score, an action plan, ready fixes and 20 diagnostic scripts.

SkillExtended access103

README in short

The repository consists of a dataset, a small classifier and a skill. The authors published it because ready lists matched User-Agent substrings and believed whatever a request claimed.

README badge

Original SKILL.md

---
name: agent-traffic
description: Classify HTTP traffic as agents, AI crawlers, search crawlers, bots or humans using the agent-traffic dataset. Use when adding agent or bot detection to request handlers, middleware, analytics, server logs or SQL queries, or when someone asks how much of their traffic comes from agents.
---

# Classify agent traffic

[agent-traffic](https://github.com/mintlify/agent-traffic) classifies HTTP requests as agents, AI crawlers, search crawlers, bots or humans. The dataset and rules are already written. Use them for whatever the user asks, whether that's classifying requests, analyzing logs or answering questions about their traffic, without changing what the rules mean.

## Sources

Read these before writing anything:

- [`GUIDELINES.md`](https://github.com/mintlify/agent-traffic/blob/main/GUIDELINES.md) has the categories, decision order and verification rules.
- [`data/agents.json`](https://github.com/mintlify/agent-traffic/blob/main/data/agents.json) lists known agents and crawlers: user agent patterns, `Accept` and `Signature-Agent` fingerprints, verification sources and real example user agents.
- [`data/http-clients.json`](https://github.com/mintlify/agent-traffic/blob/main/data/http-clients.json) lists generic HTTP libraries and headless browsers.

If the package is installed, the same data is at `node_modules/@mintlify/agent-traffic/dist/data/`. Every pattern is a case-insensitive regular expression.

## 1. Find out what's available

Before choosing an approach, look at the user's code or schema and confirm:

- **Where classification can run:** in request-handling code, in an ingest pipeline, or only over stored logs.
- **Which fields exist:** the user agent, the `Accept` header, other request headers, and the client IP. Also check whether that IP is the real client or a proxy or CDN address.
- **What they want out:** a single label, per-agent breakdowns, or verified-only numbers.

Every result needs at least the user agent. The `Accept` header and the real client IP make it much more accurate. If the user can add fields to their logs, recommend logging `Accept` and the real client IP.

## 2. Pick the approach

### JavaScript or TypeScript at request time

Install the package and call `classify` where requests are handled. Don't reimplement the rules.

```
npm i @mintlify/agent-traffic
```

```js
import { classify, loadIpRanges } from '@mintlify/agent-traffic'

let ipRanges

async function classifyRequest(request) {

Report a problem