AI-Infra-Auto-Driven-SKILLS
Ten skills for LLM serving: SGLang and vLLM benchmarks, profiler triage, memory planning and PR history.
- 1,168 ★ on the repository
- Last commit
Minimal access
Why this level
- Benchmarks and profiling run heavy GPU workloads, so watch resource use.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/ai-infra-auto-driven-skillsDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewMinimal accessThe tool runs nothing and makes no network calls.
- Benchmarks and profiling run heavy GPU workloads, so watch resource use.
This is third-party code. Review the repository files before installing.
What it does
A set of ten skills as plain `SKILL.md` folders gives an agent working memory for AI infrastructure tasks: model serving in SGLang, vLLM, TensorRT-LLM and TokenSpeed. The skills compare frameworks under one workload, the same GPUs and a shared SLA, find the best launch command for a model, and explain startup memory, KV cache budget and OOM causes from logs. Other skills read torch profiler traces and produce kernel, overlap and fusion tables checked against a catalog of known optimizations, split a trace into passes, layers and kernels with Perfetto ranges, and estimate operator shapes, FLOPs and MFU. Separate skills turn a new model architecture into a PR plan for Day-0 SGLang support, triage queue growth, timeouts and stalls in production, return public architecture diagrams of popular models and make an explainer video from an SGLang PR. The `model-pr-optimization-history` directory holds 118 bilingual dossiers: SGLang 45, vLLM 44, TensorRT-LLM 15, TokenSpeed 14. Each lists the implementation files of a model family and every PR that changed them. Search runs with `python3 scripts/query.py --framework vllm "qwen3 fused qk norm"`.
Who it is for. Inference and ML infrastructure engineers who tune LLM deployments and chase GPU bottlenecks with an agent's help.
Good fit when
- You are choosing between SGLang and vLLM for a model and need a comparison run under equal conditions.
- A server ran out of memory at startup and you must read the KV cache budget from logs.
- You have a profiler trace and need to find kernels where operations can be fused.
- A new model architecture ships and you need a support plan for SGLang.
- Before patching a model path you want to see which PRs already solved a similar task.
Not a fit when
- The tasks are not about LLM serving, since the skills are narrowly tuned to SGLang, vLLM, TensorRT-LLM and TokenSpeed.
- You have no GPU access, while benchmarks and profiling need real hardware.
- You need general ML knowledge with no tie to serving frameworks.
Example request
Use llm-serving-auto-benchmark to find the best launch command for our model in SGLang and vLLM on 8 GPUs under a 200 ms SLA.Limitations
Performance claims are tied to a specific model, hardware, precision, workload and framework revision and cannot be carried to other conditions. New PR history entries are marked as source inventories awaiting manual diff review. The video skill needs HyperFrames and edge-tts. The license is not detected in GitHub metadata, so check the LICENSE file. Some texts are in Chinese. Some skills assume a GPU server rather than a laptop.
How to disable. In Claude Code run `/plugin uninstall ai-infra-auto-driven-skills`. For a manual install delete the skill symlinks from the agent's skills directory.
FAQ
Which frameworks are covered?
SGLang, vLLM, TensorRT-LLM and TokenSpeed, with some skills touching FlashInfer and SGLang diffusion models.
What is in the PR history?
118 bilingual dossiers by model family listing implementation files and every PR that changed them.
How do I search the PR history?
With `python3 scripts/query.py --framework vllm "qwen3 fused qk norm"` in the `model-pr-optimization-history` directory.
How are conclusions checked?
Benchmark rows store model, framework commit, GPUs, workload, SLA and raw artifacts, and profiler reports keep prefill and decode separate.
Does it work outside Claude Code?
Yes, the skill folders can be linked into any agent's directory: Claude Code, Codex, Kimi and others.
Discussion
No questions or reviews yet.
Related
SkillsA set of eight skills for AI infrastructure: TileLang GPU kernels, Megatron memory, SLIME training and diagrams.
AI Research Skills
AI Research Skills Library
98 skills for ML research, from ideation and model training to inference, evaluation and paper writing
A skill that asks ChatGPT and Gemini before Claude shows its plan, weighs the three views and delivers one plan crediting each model's contribution
agentperf-local measures how fast your machine serves an AI agent on recorded conversations: throughput and latency
Prompt caching audit
LLM Cache Audit Skill
Skill for Codex and Claude Code that traces what breaks LLM prompt caching in code and config and proposes a verification step
Ten MLflow skills teach a coding agent to add tracing, diagnose failures and evaluate GenAI apps
README in short
The authors present the set as operational memory for AI infrastructure: fair benchmarks, profiler triage, model support plans and 118 PR dossiers. It includes evidence rules for performance claims.