ML Engineering Skills
Two skills from Stas Bekman's book: training large models on a cluster and an acceptance run of a GPU cluster with a report.
- 19,674 ★ on the repository
- Last commit
Extended access
Why this level
- `evaluate-cluster` downloads scripts and runs load tests on cluster nodes.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/ml-engineering-skillsDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- `evaluate-cluster` downloads scripts and runs load tests on cluster nodes.
This is third-party code. Review the repository files before installing.
What it does
The repository of the open book Machine Learning Engineering keeps agent skills in the `skills` folder. `ml-engineering` condenses the experience of training BLOOM-176B and IDEFICS-80B into a working guide: choosing accelerators, storage and network, SLURM and Kubernetes, throughput and memory, instabilities and NaN, checkpoints, multi-node hangs, inference. `evaluate-cluster` runs acceptance of trial nodes: environment dump, matmul FLOPS, intra- and inter-node all-reduce bandwidth, `fio` on local and shared disks and a dated Markdown report.
Who it is for. ML engineers and GPU cluster administrators who train or serve large models and choose hardware and cloud.
Good fit when
- Training shows low MFU and the bottleneck must be found: compute, memory, network, disk or dataloader.
- A loss spike or NaN appeared mid-run.
- A multi-node job hangs on NCCL.
- A cloud provider gave trial nodes that must be checked before buying.
- Inference is sized by latency and throughput.
Not a fit when
- The task is an ordinary application on a single GPU with no distributed training.
- A beginner tutorial on machine learning basics is needed.
- The cluster belongs to someone else and benchmarks may not run on it.
Example request
Evaluate a 2-node trial cluster: measure FLOPS, all-reduce and disk, then compile a report with conclusions.Limitations
The skills are in English and condense the book, so for detail and scripts the agent follows links to chapters on GitHub. `evaluate-cluster` requires checking against the GitHub copy, downloads scripts and then runs load tests on your cluster, so confirm that beforehand. Advice mostly concerns NVIDIA and AMD GPUs and large clusters. The material is licensed CC BY-SA 4.0. Performance numbers depend on hardware generation, so fresh measurements on your own cluster outweigh tables from the book.
How to disable. Delete the `ml-engineering` and `evaluate-cluster` folders from the agent skills directory. Remove downloaded benchmark scripts and reports separately.
FAQ
How do I add a skill?
Copy the needed `SKILL.md` into the agent skills directory or point the agent at the GitHub file. The root `SKILL.md` only redirects to `skills/ml-engineering/SKILL.md`.
What is included?
Two skills: `ml-engineering` for training and operating large models and `evaluate-cluster` for GPU cluster acceptance.
What does evaluate-cluster measure?
Matmul FLOPS on every card, all-reduce within and across nodes and `fio` on local and shared disks. The result is a dated Markdown report.
Does it help debug hangs?
Yes, `ml-engineering` covers multi-node hangs. For debugging technique such as gdb and py-spy the author suggests adding The Art of Debugging skill.
What license covers the materials?
CC BY-SA 4.0. Publish copies and derivative works under the same license with attribution.
Discussion
No questions or reviews yet.
Related
SkillsA global intelligence MCP server: news, geopolitical risk, markets and infrastructure exposed as agent tools
166 skills for scientific work: bioinformatics, cheminformatics, clinical data, geospatial analysis and 100+ databases
The native ClickHouse client executes SQL and exports results for agent-prepared analytical queries reviewed by a user.
Streamlit agent skill
developing-with-streamlit
The official Streamlit skill bundled with the library: building, styling, debugging and speeding up apps, installed with streamlit skills
migrate CLI applies versioned database changes in order, with a chosen step count or target schema version for each run.
Google's open-source MCP server for databases: ready tools for Postgres, MySQL, BigQuery, Spanner and more, plus custom tools in tools.yaml
README in short
`skills/README.md` lists the two book skills and suggests copying a `SKILL.md` into the agent directory or pointing to the GitHub link.