LangSmith skills

Install

npx skills add langchain-ai/langsmith-skills --skill '*' --yes

This is third-party code. Review the repository files before installing.

What it does

Three skills help you observe and evaluate LLM apps. langsmith-trace adds tracing to code and exports traces for debugging through the langsmith CLI. langsmith-dataset builds evaluation datasets from traces. langsmith-evaluator creates custom evaluators. Each skill ships Python and TypeScript helper scripts.

Who it is for. For developers of LLM apps and agents who use LangSmith.

Example request

Export yesterday's failing traces from the support-bot project and turn them into a dataset

Limitations

Early development; content may change. Requires a LangSmith key, plus OpenAI or Anthropic keys for models. The trace skill needs the langsmith CLI installed.

How to disable. Uninstall the langsmith-skills plugin via /plugin or delete the skill folders from the agent's directory.

Security check

• Runs helper scripts and a CLI
• Reads traces that may contain user data

README in short

The README warns about early development and points to langchain-skills for agent building. It covers three install paths: skills.sh locally or globally, a Claude Code plugin and an install.sh script for Claude Code and Deep Agents CLI. Then it lists env vars and the three skills.

SKILL.md

---
name: langsmith-trace
description: "INVOKE THIS SKILL when working with LangSmith tracing OR querying traces. Covers adding tracing to applications and querying/exporting trace data. Uses the langsmith CLI tool."
---

<oneliner>
Two main topics: **adding tracing** to your application, and **querying traces** for debugging and analysis. Python and Javascript implementations are both supported.
</oneliner>

<setup>
Environment Variables

```bash
LANGSMITH_API_KEY=lsv2_pt_your_api_key_here          # REQUIRED
LANGSMITH_PROJECT=your-project-name                   # Optional: default project
LANGSMITH_WORKSPACE_ID=your-workspace-id              # Optional: for org-scoped keys
```

Authentication is REQUIRED: either set the `LANGSMITH_API_KEY` environment variable, or pass the `--api-key` flag to CLI commands (preferred):
```bash
langsmith trace list --project my-project --api-key $LANGSMITH_API_KEY
```

**IMPORTANT:** Always check the environment variables or `.env` file for `LANGSMITH_PROJECT` before querying or interacting with LangSmith. This tells you which project contains the relevant traces and data. If the LangSmith project is not available, use your best judgement to identify the right one.

CLI Tool
```bash
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh
```
</setup>

<trace_langchain_oss>

Related