SkillEvaluator CLI
skillevaluator
An NVIDIA framework for evaluating agent skills: quality checks, overlap detection, synthetic checks and live evaluation by an agent. Experimental
- 564 ★ on the repository
- Last commit
- Apache-2.0
Extended access
Why this level
- An experimental 0.x project
- Live evaluation spends tokens on your key
Install
- Works in
Manual install
git clone https://github.com/NVIDIA/SkillEvaluator.gitThen follow the README. Live evaluation needs a model key (NVIDIA Build or OpenAI-compatible).
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- An experimental 0.x project
- Live evaluation spends tokens on your key
This is third-party code. Review the repository files before installing.
What it does
SkillEvaluator evaluates agent skills in several tiers: deterministic quality checks, overlap detection between skills, synthetic check sets and live evaluation by running an agent. A model key is needed (NVIDIA Build or OpenAI-compatible). Installation is through `git clone https://github.com/NVIDIA/SkillEvaluator.git`. Per the README the support level is experimental and community maintained. Version 0.4.0 from 30.09.2026, Apache-2.0 license.
Who it is for. For authors and teams who write many skills and want to measure their quality and find duplicates.
Good fit when
- You need to check the quality of your skills automatically
- You need overlap detection between skills
- You need live evaluation of a skill by running an agent
Not a fit when
- You need stable support: the project is experimental
- You have no model key: some tiers do not work without it
Example request
Evaluate my skill set, find the ones that overlap in purpose and show which checks failedLimitations
The support level is experimental, version 0.x. Live evaluation runs an agent on your skills and spends tokens on your key, so access is extended. Start with deterministic checks, which need no key.
How to disable. Delete the repository clone and revoke the model key where it was issued.
FAQ
What does SkillEvaluator evaluate?
Skill quality, overlaps, synthetic checks and the result of a live agent run.
Is a key needed?
For live evaluation yes: NVIDIA Build or OpenAI-compatible.
How mature is the project?
Experimental, version 0.4.0.
Discussion
No questions or reviews yet.
Related
CLI toolsA CLI to evaluate and evolve agent skills: declarative YAML cases, runs across engines and structured reports
Unit tests for skills: checks that an agent finds and uses your skill. Node 20 and Docker are needed, plus a model key
Swift Testing Agent Skill
swift-testing-agent-skill
A Swift Testing skill from Antoine van der Lee: the new APIs, migration from XCTest, parameterized tests and flat tests. For Claude Code and other agents
QA agents, skills and an MCP server: test generation, coverage gaps, flaky tests, security scans and chaos testing
A skill that lets an agent write and run Playwright scripts: page tests, forms, screenshots and link checks
Tester
Тестер
A scenario testing system for 1C:Enterprise with a set of agent skills that let an AI agent operate the application like a regular user
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Repository stars | Main difference |
|---|---|---|---|
| SkillEvaluator (this entry) | yes | 564 | |
| SkillCorpus | no | 685 | a pipeline for collecting, checking and matching skills |
| upskill | yes | 762 | a strong model builds a skill from traces and a weak one uses it |
| skillgrade | no | 724 | unit tests for skills in containers |
README in short
The README describes a skill evaluation framework with deterministic checks, overlap detection and live evaluation. Apache-2.0 license.