Tracely skill

---
name: tracely
description: Instrument AI agents with Tracely and turn their production traces into CI gates. Use when the user mentions Tracely, tracelyai-sdk, tracely_sdk, the `tracely` CLI, or asks to trace/observe an AI agent, add LLM evaluators or LLM-as-a-judge columns, debug why a trace or conversation isn't showing up, wire agent regression tests into a PR check, run scenario or red-team suites against an agent endpoint, or replay recorded agent failures in CI. Covers both zero-span-code automatic instrumentation and the manual span API.
license: MIT
---

# Tracely

Trace-native CI/CD for AI agents. One loop:

```
production trace → failure detection → regression test → CI gate
```

The trace is the source of truth. Evaluators, failure clusters, regression cases, gates and trends
are all **derived from it** — there are no hand-authored datasets. Docs: <https://doc.tracely-ai.com>.

## Pick the path first

Do not start writing spans. Ask what the code already looks like, then pick:

| The user's code | Path | Effort |
|---|---|---|
| Calls OpenAI / Anthropic / Gemini / Mistral / Bedrock / Groq SDKs | **Automatic** — `init(instrument="auto")` | 1 line |
| Uses LangChain / LangGraph / LlamaIndex / CrewAI / LiteLLM | **Automatic** + that extra | 1 line |
| Uses OpenAI Agents SDK / Claude Agent SDK / Google ADK | **Automatic**, named explicitly | 1 line |
| Has business logic worth seeing (routers, tools, retrievers) | Automatic **+ `@observe`** | 1 decorator each |
| Needs spans the auto path can't produce (custom retrievers, guardrails, handoffs, multimodal I/O, hand-rolled providers) | **Manual context managers** | `references/manual.md` |
| Is TypeScript / Go / Ruby / anything not Python | **Emit OTLP directly** — no SDK | `references/automatic.md` § other languages |
| Already emits OpenTelemetry / OpenInference / OpenLLMetry | Point the existing exporter at Tracely, add 2 attributes | `references/automatic.md` § other languages |

**Default to automatic.** Manual spans are the escape hatch, not the starting point, and the two
compose — manual spans nest inside auto-instrumented traces in the same tree.

## Connect

```python
import tracely_sdk as tracely   # pip install "tracelyai-sdk[openai]"  ([anthropic] [langchain] [all])

tracely.init(
    endpoint="http://localhost:8000",   # hosted: https://api.tracely-ai.com
    api_key="tracely_dev_key",          # an ingest key — Settings → API keys. The key IS the workspace.
    service_name="support-agent",
    env="prod",                         # prod | staging | ci | dev — the gating axis
    instrument="auto",
)
```

Python ≥ 3.10. Import name is `tracely_sdk`; the CLI is `tracely`. `init()` is idempotent — call it
once at startup. Prefer `os.environ` for `endpoint`/`api_key` in real code; never inline a key.

## The 90% path

```python
with tracely.trace(agent="support-agent", conversation="conv-1", user="u_42"):
    client.chat.completions.create(model="gpt-4o", messages=[...])   # traced, no span code
```

`trace()` opens no span — it stamps run context (`agent`, `conversation`, `turn`, `user`, `env`,
arbitrary metadata) onto **every** span inside it, including the ones the instrumentor created.