Waza CLI
Microsoft's official CLI runs graded trials for skills and agents, checking the evidence behind a result, not just pass or fail.
- 1,509 ★ on the repository
- Last commit
- MIT
Extended access
Why this level
- Authenticated workflow evaluation makes real, sometimes paid, calls to services once explicitly opted in.
Install
- Works in
Manual install
Install the published `v0.40.0` binary release from the Releases page, or run `waza update`. Start with the offline harness (the Quick Start in the docs), then move to authenticated workflow evaluation following getting-started.
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- Authenticated workflow evaluation makes real, sometimes paid, calls to services once explicitly opted in.
This is third-party code. Review the repository files before installing.
What it does
Waza is a Go CLI for evaluating agent workflows, skills and custom agents: it authors scenarios, runs graded trials and lets you inspect the evidence behind a decision rather than just a pass-or-fail flag. The tool covers scenario authoring, preflight checks, selected-file evidence, assurance profiles and controlled release policy. The project draws a clear line between rigor levels: a mock-based check ("mock success") verifies the harness itself, not agent quality or live-service safety; a real check needs an authenticated run with real calls. Installation uses a prebuilt binary release or `waza update`; building from source is possible but a separate path, because LFS artifacts block `go install`.
Who it is for. Teams publishing agent skills or MCP scenarios who want a graded trial with evidence, not a simple yes-or-no check.
Good fit when
- Evaluating a skill or agent's quality by a graded trial, not a binary pass-fail.
- CI needs regression gates based on fixed scenarios and recorded evidence.
- Paid calls need explicit limits and user consent before running, not a silent budget drain.
- An agent's decision evidence needs breaking down by specific files, not just trusting the final summary.
Not a fit when
- No real run is wanted at all: the offline harness does not actually evaluate a live service or agent quality.
- Installing via `go install` is required: LFS artifacts block that, so a binary release or a repository clone is needed instead.
- The public "Ask Waza" chat assistant is expected to work: as of the README, the public service is not yet configured.
Example request
Run a graded trial of this skill with Waza's offline harness and show the file-level evidence for the failing test.Limitations
Installing with `go install` does not work because of LFS artifacts; a published binary release, `waza update`, or cloning a specific repository tag for a source build is needed instead. Authenticated evaluation of workflows with paid calls needs explicit opt-in and the limits described separately in the platform's feature matrix. The public "Ask Waza" chat assistant is unavailable as of the README, until a public service is configured; only a separately launched, loopback-only private preview is available.
How to disable. Remove the installed `waza` binary and the cloned repository, if it was used for a source build.
FAQ
Why doesn't go install work?
Because of LFS artifacts used in the repository; a published binary release needs installing instead, or a specific tag cloned for a source build.
What does mock success mean?
It is a passing check of the evaluation harness itself on synthetic data, not confirmation of agent quality or live-service safety.
Is the public Ask Waza chat assistant available?
Not as of the README: the panel explains the public service is not yet configured, and only a separate loopback-only private preview exists.
Is explicit consent needed for paid calls?
Yes, authenticated evaluation with paid calls needs opt-in and following the limits in the platform's feature matrix.
Which CLI version is current?
The published standalone version `v0.40.0` as of 4 November 2026, carrying `v0.39.0`'s features (scenario authoring, preflight, evidence, assurance profiles).
Discussion
No questions or reviews yet.
Related
CLI toolsA CLI that tests AI agents end to end: derives scenarios from code, runs them live and separates unverified from failed.
Vercel CLI for testing coding agents on your framework: runs Claude Code, Codex and others and measures the success rate
CLI checks a web page in Chromium: finds every control, uses it and reports what broke. No tests to write and no LLM.
The CLI runs QA Wolf flows locally and in CI, refreshes cached tests and helps investigate traces from failed attempts.
Behavioral eval lab Quorum for the superpowers project: drives real agent CLIs and grades them against private criteria
QA agents, skills and an MCP server: test generation, coverage gaps, flaky tests, security scans and chaos testing
README in short
The documentation splits the user path into four steps: concepts and learning order, an offline harness for a quick start, authenticated workflow evaluation, and CI regression gates.