Agent Reliability Analyzer plugin
Agent Reliability Analyzer statically scans AI agent code, finds reliability gaps with 340 plus rules and applies fixes
- 88 ★ on the repository
- Last commit
- Apache-2.0
Extended access
Why this level
- The enrich command with `--apply` changes repository source files
- For fixes, code fragments go to the chosen model provider with your key
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add plugins/agent-reliability-analyzerDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- The enrich command with `--apply` changes repository source files
- For fixes, code fragments go to the chosen model provider with your key
This is third-party code. Review the repository files before installing.
What it does
The Trustabl scanner parses an agent repository and looks for what breaks agents in production: vague tool descriptions from which a model cannot tell when to call them, missing retries and timeouts, untyped parameters, absent guardrails and tool grants wider than the claimed task. Per the docs there are 340 plus rules, nine SDKs and seven languages. Scanning runs locally: no cloud scanner, no account, no code upload and no LLM. Results come as text, JSON or SARIF 2.1.0, and exit codes suit CI. The `trustabl enrich --diff --apply` command shows a patch and applies the fix to the source, which needs your own Anthropic, OpenAI or Google key. Deepest coverage goes to the Claude Agent SDK, OpenAI Agents SDK, Google ADK and MCP servers, plus LangChain, LangGraph, CrewAI, AutoGen, Pydantic AI and the Vercel AI SDK. The Claude Code plugin adds commands such as `/trustabl:scan`, downloads the pinned binary on first run and checks its checksum against the release `checksums.txt`. The scanner also runs as a local MCP server, `trustabl mcp`.
Who it is for. Teams building their own agents on the Claude Agent SDK, OpenAI Agents SDK or MCP who want to catch reliability gaps before release.
Good fit when
- A Google ADK agent fails in production for lack of timeouts and they must be found across the repository
- CI needs a SARIF report for a repository with an MCP server and subagents
- Tool descriptions are too vague and the model confuses which to call
- Fixes should first be seen as a patch and only then applied
Not a fit when
- You need to measure the agent at runtime: the scanner is static and does not watch an agent run
- You need prompt injection protection on the fly: the tool reads code and does not intercept agent calls
- The agent uses an SDK missing from the COVERAGE.md table
Example request
Scan this repository with Trustabl, show findings for the agent tools and propose a patchLimitations
- The scanner is static: it does not check agent behavior at runtime.
- Applying fixes through `trustabl enrich` uses your Anthropic, OpenAI or Google key, and code fragments go to that provider.
- On first run the plugin downloads a binary for your platform and runs it after the checksum matches.
- SDK and language coverage is uneven, see `COVERAGE.md`.
- The plugin needs the network on first run: the binary downloads from the release.
How to disable. Remove the plugin through `/plugin` in Claude Code. A Homebrew installed binary is removed with `brew uninstall trustabl`.
FAQ
Does my code go to the cloud on a scan?
No, scanning is local, with no account and no LLM. Code goes to a provider only on `trustabl enrich`, with your key.
How do I run it without installing?
With `docker run --rm -v "$PWD:/repo" ghcr.io/trustabl/trustabl:latest scan /repo`.
Which SDKs does it cover?
Deepest are the Claude Agent SDK, OpenAI Agents SDK, Google ADK and MCP servers, plus LangChain, LangGraph, CrewAI, AutoGen, Pydantic AI and the Vercel AI SDK.
Which output formats does the report use?
Human readable text, JSON and SARIF 2.1.0. Exit codes suit CI checks.
How do I install the plugin in Claude Code?
Run `/plugin marketplace add trustabl/claude-plugin` or find trustabl in Plugins, Discover. Then run `/trustabl:scan`.
Discussion
No questions or reviews yet.
Related
PluginsA local debugger for TypeScript agents: run step trees, CI trajectory checks and portable evidence bundles, with no account.
A local web dashboard for agent reliability scans, skill generation and network probes with no separate tool installs
Vercel CLI for testing coding agents on your framework: runs Claude Code, Codex and others and measures the success rate
CLI installs a verify-by-running harness into Claude Code and Codex: build and tests, diff review and a separate QA agent.
A check for AI-generated code: empty functions, imports of things that do not exist, and docs that oversell.
An MCP server that checks whether the agent is really done: drives the real app and returns pass, fail or couldn't tell with the file and line
Comparison with similar tools
Cards that share the main search query, and how they differ.
| Tool | Official | Repository stars | Main difference |
|---|---|---|---|
| Agent Reliability Analyzer (this card) | no | 88 | |
| AgentLint | no | 156 | a live guard that blocks secret writes and risky commands and does not scan agent code |
| agent-eval | yes | 280 | measures the task success rate by running agents, the scanner reads code only |
README in short
The README presents the project as a deterministic agent reliability scanner, shows install through Docker, Homebrew and Scoop, output formats, a real report sample and the supported SDK list. Integrations are described separately: the Claude plugin, VS Code, Cursor, GitHub Actions and the MCP registry.