skill-up CLI
A CLI to evaluate and evolve agent skills: declarative YAML cases, runs across engines and structured reports
- 1,471 ★ on the repository
- Last commit
- Apache-2.0
Extended access
Why this level
- The CLI runs skills on agent engines and executes grading scripts
- Runs and the agent judge call the model and spend tokens
Install
- Works in
Manual install
curl -fsSL https://raw.githubusercontent.com/alibaba/skill-up/main/install.sh | bashInstall the skill-up CLI directly. Usually skill-upper installs the CLI itself on first run.
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The CLI runs skills on agent engines and executes grading scripts
- Runs and the agent judge call the model and spend tokens
This is third-party code. Review the repository files before installing.
What it does
skill-up is a CLI that makes skill quality measurable and repeatable. Test cases are declared in YAML, run across multiple agent engines and graded by rules, scripts or an agent judge, with results collected into reports locally or in CI. It ships the skill-upper skill: through conversation it reads failures, repairs and expands the case set, reruns skill-up and repeats the loop. It imports Anthropic's evals.json format and produces compatible grading.json and benchmark reports.
Who it is for. For developers and QA engineers who build agent skills and want to test them with regressions.
Good fit when
- You need to measure skill quality with tests, not by eye
- You need regression cases for a skill in CI
- You want to move an Anthropic evals.json into a repeatable run
Not a fit when
- You do not build skills, you only use them
- You have no access to an agent engine to run the cases
Example request
Write an eval for my skill and run it through skill-up, then show the failure reportLimitations
Running cases needs an agent engine: Claude Code, Codex and Qoder CLI are built in, plus custom engines. Agent-judge grading and the runs themselves spend model tokens of whichever engine you connect.
How to disable. Remove the skill-up binary from PATH and the skill-upper folder from the agent's skills directory.
FAQ
Do I install skill-up before skill-upper?
Usually no. skill-upper checks for the CLI at runtime and guides the agent through installing it if missing.
How are answers graded?
By three strategies: rule based, script and agent judge, with results collected into structured reports.
Discussion
No questions or reviews yet.
Related
CLI toolsValidates Agent Skills packages: structure, links, token weight and instruction quality, with optional LLM-as-judge scoring
Unit tests for skills: checks that an agent finds and uses your skill. Node 20 and Docker are needed, plus a model key
A Python CLI that measures what a skill changes: runs a case with and without it, lints answer leaks in prompts and diffs reports
A CLI and MCP for testing agent skills: repeatable YAML evals, report comparison, regression detection and skilled-improve
Caliper runs agent tasks with and without your skill, then shows whether the skill helps and how many tokens it costs
SkillEvaluator
skillevaluator
An NVIDIA framework for evaluating agent skills: quality checks, overlap detection, synthetic checks and live evaluation by an agent. Experimental
README in short
The README describes skill-up as an evaluation and evolution tool for skills, written in Go. Evaluation is set through eval.yaml and case files, runs on Claude Code, Codex and Qoder CLI engines plus custom ones, and grading can be rule, script or agent based. Reports are Anthropic compatible, with evals.json import and CI modes. The recommended path is the skill-upper skill, which repairs and expands cases and reruns the loop through conversation. Apache-2.0 licensed.