agentsre skill
A skill and Python library: four agent decision-quality indicators, a repeated-write guard and a circuit breaker for agent chains
- 309 ★ on the repository
- Last commit
- MIT
Extended access
Why this level
- Publishing metrics to CloudWatch uses the AWS access of the environment where the agent runs.
- The library lives in production code and blocks calls by its own thresholds, so a wrong setup can stop working operations.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/agentsreDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- Publishing metrics to CloudWatch uses the AWS access of the environment where the agent runs.
- The library lives in production code and blocks calls by its own thresholds, so a wrong setup can stop working operations.
This is third-party code. Review the repository files before installing.
What it does
The `agentsre` skill teaches a coding agent to wire the agentsre library into a production AI system. Ordinary metrics stay green even when an agent makes wrong decisions: the response is 200 and latency is normal. The library computes four semantic-level indicators: decision quality (DQR), tool calls per task (TIE), human escalation rate (HER) and approval queue depth drift (AQDD). It also includes `IdempotencyGuard`, which blocks a retried write that already succeeded, the agent-to-agent output validator `A2ASemanticValidator`, the `AgentChainCircuitBreaker` circuit breaker and an agent fleet inventory module with model deprecation dates. Metrics can go to AWS CloudWatch through the `agentsre[aws]` extra.
Who it is for. SRE engineers and developers who take agent systems to production and own their reliability.
Good fit when
- Monitoring is green while the share of wrong agent decisions grows, and complaints are the only signal.
- An agent repeats a write such as a funds transfer and the duplicate has to be blocked.
- One agent's result goes to another and needs a schema and quality check before handoff.
- An agent framework upgrade is rolled out with a canary check of TIE and DQR drift.
Not a fit when
- You only need traces of individual model requests, while the library computes indicators from task records.
- Duplicate protection is needed across processes: `IdempotencyGuard` works inside one, so put a conditional write in a shared store in front of it.
- The project is not written in Python.
Example request
Wire agentsre into the payment routing service and set an alert when TIE grows one and a half times.Limitations
The library is Python with a dependency-free core; CloudWatch needs the `aws` extra with boto3. The application supplies `decision_confidence` itself, and without it DQR cannot be computed. The first `collect()` call has no baseline, so drift appears from the second. `IdempotencyGuard` works within one process. The alert thresholds from the README (for example, DQR below 85 percent) should be tuned to your own system.
How to disable. Remove the `agentsre` skill folder from the agent skills directory and run `pip uninstall agentsre`. Remove the library calls in project code separately.
FAQ
Which four indicators does agentsre compute?
DQR shows decision quality, TIE counts tool calls per task, HER measures the share of escalations to a human and AQDD tracks growth of the approval queue.
How do I send metrics to CloudWatch?
Install `pip install agentsre[aws]` and pass the `collector.collect(...)` results to `CloudWatchPublisher`. Metrics land in the `AgentReliability` namespace.
What does IdempotencyGuard do?
It blocks a repeated call of an already executed write within the `window_seconds` window, such as a funds transfer, and exposes the `duplicate_action_rate` metric. The guard works inside one process.
When do alerts fire?
Per the skill table: DQR below 85 percent, TIE at 1.5 times the baseline or more, HER above 5 percent, and AQDD when the queue grows against the baseline.
Does the library need dependencies?
The core has none. Only publishing to CloudWatch needs `agentsre[aws]`, which adds boto3.
Discussion
No questions or reviews yet.
Related
SkillsForty-five skills, custom agents and CloudFormation templates for AWS DevOps Agent: incident analysis and operations reviews
ANOLISA is a system stack for AI agents: output compression, memory, workspace snapshots, command checks and token accounting
Eight skills for Azure SRE Agent audit architecture, policy compliance, capacity and cost of a subscription and score it
Microsoft Agent Skills
Agent Skills
Official Microsoft skills that ground coding agents in Azure SDKs and Foundry for Python, .NET, TypeScript, Java and Rust
A large base of DevOps, security, infrastructure and compliance skills for AI agents
AWS's official set of MCP servers, skills and plugins so an agent can build, deploy and operate on AWS
README in short
The library adds semantic-level indicators to availability metrics, validation of agent-to-agent output and a circuit breaker based on the validation success rate. The repository includes LangChain and fleet governance examples.
README badge
Original SKILL.md
---
name: agentsre
description: Instrument AI agents with SRE reliability signals using the agentsre Python library. Use when adding semantic SLIs (decision quality, tool-call efficiency, human escalation, approval queue drift), blocking duplicate irreversible agent actions, validating agent-to-agent output, or adding a circuit breaker to a multi-agent chain.
---
# agentsre
Reliability instrumentation for agentic AI in production. Standard SLIs (HTTP 200, latency,
error rate) stay green while an agent makes wrong decisions. agentsre measures the semantic layer.
Install: `pip install agentsre` (core has no dependencies; `agentsre[aws]` adds CloudWatch publishing).
## The four SLIs
| SLI | Meaning | Alert when |
|---|---|---|
| DQR | Decision Quality Rate (mean confidence of completed tasks) | below 85% |
| TIE | Tool Invocation Efficiency (tool calls per task vs. baseline) | 1.5x baseline or more |
| HER | Human Escalation Rate | above 5% |
| AQDD | Approval Queue Depth Drift | queue growing vs. baseline |
## Record tasks and read SLIs
```python
from agentsre import AgentSLICollector, TaskRecord
collector = AgentSLICollector()
collector.record(TaskRecord(
task_id="t-001", task_class="payment-routing",
tool_calls=3, required_escalation=False, pending_approval=False,
decision_confidence=0.91, completed=True,
))
for result in collector.collect("payment-routing"):
print(result)
if collector.breached("payment-routing"):
... # page or degrade
```
Drift is measured against the baseline from earlier `collect()` calls, so call `collect()`
on a schedule. The first call has no baseline and reports `drift_ratio=None`.
## Block a retried write
```python
from agentsre import IdempotencyGuard
guard = IdempotencyGuard(window_seconds=300, max_repeats=1)
decision = guard.check("transfer_funds", {"to": "acct-9", "amount": 120}, scope="user-42")
if not decision.allowed:
... # do not repeat the write