Prompt caching audit skill

LLM Cache Audit Skill

Skill for Codex and Claude Code that traces what breaks LLM prompt caching in code and config and proposes a verification step

  • 130 ★ on the repository
  • Last commit
  • MIT
Skill

Extended access

Why this level

  • Runs local Python scripts for analysis
All reasons and checks

sernote/audit-prompt-caching

Install

Services

Manual install

npx skills add https://github.com/sernote/audit-prompt-caching --skill audit-prompt-caching

Run from the project you want to audit. Requires Node.js.

Access and checks

How we review

Extended accessThe tool runs code or makes network calls and only reads external data.

  • Runs local Python scripts for analysis

This is third-party code. Review the repository files before installing.

What it does

The agent walks the LLM request path: prompt assembly, message order, tools and schemas, provider settings and routing. An audit ends in one of three results: a finding with evidence and a verification step, a justified no-change conclusion, or a list of missing measurements. Reference files cover OpenAI, Anthropic, Gemini, Bedrock, OpenRouter, vLLM, SGLang, YandexGPT and other providers. Local Python scripts compare prefixes, lint payloads and estimate ROI.

Who it is for. Developers of LLM apps and agents whose cost or latency went up while cached_tokens stays at zero.

Good fit when

  • cached_tokens or cache_read_input_tokens is zero on repeated requests
  • Cost rose after changes to tools, schemas or the agent loop
  • You need to decide whether caching is worth optimizing at all
  • KV reuse dropped after scaling vLLM or SGLang

Not a fit when

  • General prompt quality work without cost or latency concerns
  • Token counting or context window sizing

Example request

Use the audit-prompt-caching skill to audit this project's LLM calls. Start with code and configuration. Show findings, evidence, and checks.

Limitations

The skill does not capture live traffic and relies on code, config and exported telemetry. The routing analyzer is experimental and has no adapter for native router logs. Claims about pricing, TTL and model support must be checked against current provider docs.

How to disable. Delete the audit-prompt-caching folder from the agent's skills directory.

FAQ

Can I install it without Node.js?

Yes, with install.sh from a local checkout: bash install.sh --source-dir . --agent claude, codex or both.

What do the local scripts need?

Python 3.10+, standard library only.

Discussion

No questions or reviews yet.

Yandex AI Studio skill for Laravel

yandex-ai-laravel skill

Russian stack

A Claude Code skill for the yandex-ai-laravel SDK: Responses API, Vision OCR across 12 models, embeddings and known differences from OpenAI

SkillMinimal access
Russian stack

A skill collection for the Russian stack: GigaChat, YandexGPT, Yandex 360, Yandex Cloud, Metrika and marketplace orders

SkillExtended access80Repository stars
Russian stack

A set of MCP servers and skills for the OpenClaw assistant: YandexGPT, OCR, speech synthesis and image generation

MCP serverExtended access1Repository stars
Russian stack

Claude Code skill for evaluating websites by Yandex EPOS criteria (expertise, usefulness, originality, content richness) and visibility in Alice AI

SkillMinimal access3Repository stars
Russian stack

Four single-file MCP servers for YandexGPT, text recognition, SpeechKit speech synthesis and Yandex search

MCP serverExtended access
Russian stack

A reference multi-agent student support assistant with an MCP server to a knowledge base and Bitrix24, over Telegram and MAX

MCP serverExtended access5Repository stars

README in short

The skill helps Codex, Claude Code and compatible agents find what breaks LLM prompt caching in a project. An audit starts from code and configuration and ties each conclusion to evidence and a verification step. The README shows a block reordering example where the shared prefix grew from 43 to 254 bytes, and a table of typical questions. Provider and self-hosted inference checks live in separate reference files. MIT license.

README badge

Original SKILL.md

---
name: audit-prompt-caching
description: "Use whenever the user mentions cached_tokens=0,total_cached_tokens,cache_read_input_tokens,cache_creation_input_tokens,cache_write_tokens,prompt_cache_key,prompt_cache_options,prompt_cache_breakpoint,previous_interaction_id,cache_control/cachePoint,TTFT,KV reuse; prefix_cache_retention_interval,prefix_caching_hash_algo,Mamba/SWA/hybrid,cross-process block hash; LLM cost or speed regressed,repeated long prompts,speeding up agents; LLM request shape changes: tools,schemas,response_format,model/router,agent loops,compaction; Not for generic prompt writing,RAG,token counts,non-LLM perf"
---

# Prompt Cache Audit

Diagnose LLM prompt/prefix cache misses as request-path engineering problems:
stable reusable prefixes, provider telemetry, cache-aware routing, and entries
that live long enough to be reused. Caching is worth changing only when the
prefix is stable, long enough, repeated, measurable, and safe. Do not add cache
controls, keys, salts, provider pins, routing hints, or broader cache sharing
until the applicability, telemetry, and trust-boundary checks justify them.

## When to use

Use this skill for LLM calls where repeated prompt prefixes may affect cost,
TTFT, prefill latency, or self-hosted KV reuse. Typical triggers:

- `cached_tokens=0`, `cache_read_input_tokens=0`, `cache_write_tokens`, writes without reads, or unclear usage fields; GPT-5.6/GPT-6 Astra `prompt_cache_options`/`prompt_cache_breakpoint`; or migration from `prompt_cache_retention`.
- Effort or reasoning changes inside a cached conversation: per-step `reasoning.effort`, GPT-6 Astra `configuration_update`, Claude per-message `output_config.effort`, thinking toggles.
- Cache hit rate, TTFT, prefill latency, or input-token cost changed; LLM cost or speed regressed around repeated long prompts, shared context, long agents, or tool loops, or a reported hit rate is not trusted.
- LLM request shape changed where repeated long prompts, TTFT, cached-token telemetry, or LLM cost matter: inspect prompt text, message order, request builders, tools, schemas, `response_format`, provider API surface, model/router settings, agent loops, or context compaction.
- Long system prompts, tool catalogs, schemas, static documents, few-shot/RAG context, provider cache APIs, or vLLM/SGLang multi-replica KV deployments with KV pressure, tokenizer/chat-template drift, cache salts, or APC benchmarks such as `vllm bench serve`, `prefix_repetition`, and `benchmark_prefix_caching.py`.

## When not to use

Do not use this skill for:
- generic prompt writing, prompt-quality editing, or ordinary short prompt edits without repeated-prefix, TTFT, cache telemetry, or LLM cost concern
- generic RAG design unless repeated context placement/cacheability is part of the task
- token counting or context-window sizing only
- response caching only, unless comparing it with prompt prefix caching
- non-LLM frontend/backend performance or non-inference Kubernetes routing
- speculative savings claims without usage data or explicitly stated assumptions

Report a problem

Foxx AIPrompt caching audit
Foxx AI

I am Foxx AI and I have already vetted this tool. Ask about install, setup or anything else, and I will keep it simple.