TruLens Skills
TruLens skills walk an agent through LLM app tracing, evaluation setup, diagnosing low scores and runtime blocking guardrails
- 3,639 ★ on the repository
- Last commit
- MIT
Minimal access
Why this level
- The skills are instructions; running evaluations spends tokens on your provider key
Install
- Works in
Manual install
pip install trulensThe first README command. Skills are copied by hand from `src/core/trulens/.agents/skills`.
Access and checks
How we reviewMinimal accessThe tool runs nothing and makes no network calls.
- The skills are instructions; running evaluations spends tokens on your provider key
This is third-party code. Review the repository files before installing.
What it does
TruLens is a library for evaluating and tracking LLM experiments and AI agents. The repository holds a set of agent skills that lead through the evaluation workflow, from instrumenting the app to reading results. The `trulens-instrumentation` skill sets up OTEL-based tracing and helps debug and optimize. `trulens-evaluation-setup` configures feedback functions and selectors, `trulens-dataset-curation` builds evaluation datasets with ground truth, `trulens-diagnosis` analyzes low scores and suggests improvements, and `trulens-blocking-guardrails` turns feedback functions into runtime blocking guardrails. A notebook execution skill is also included.
Who it is for. RAG and agent app developers who evaluate answer quality with TruLens and want an agent to guide them step by step.
Good fit when
- A RAG app gives weak answers and it is unclear whether retrieval or generation is at fault.
- Tracing has to be added to an app on OpenTelemetry.
- Evaluations need a question set with ground-truth answers.
- A groundedness score fell after a prompt edit and the causes are needed.
- Unsafe answers have to be blocked at runtime in the app.
Not a fit when
- The app does not use TruLens and there is no plan to change the evaluation tool.
- You need evaluation with no external model calls: feedback functions often call a provider.
- You want the skills as a ready package: the install path is not described in the README.
Example request
Add TruLens tracing to our RAG app, set up groundedness and context relevance evaluations and run them on twenty questions.Limitations
The skills sit in `src/core/trulens/.agents/skills` inside the sources, and the README does not describe how to install them: copy the folders you need into the agent's skills directory by hand. The library itself installs with `pip install trulens` and needs provider packages for feedback functions. Evaluations run through an LLM provider with your key, so runs spend tokens. Interactive skills ask about the app type before configuring evaluations.
How to disable. Delete the copied `trulens-*` folders from the agent's skills directory. The library itself is removed with `pip uninstall trulens`.
FAQ
How do I install the library?
Run `pip install trulens`. For evaluations add a provider package such as `trulens-providers-openai` or `trulens-providers-litellm`.
Where are the skills?
In `src/core/trulens/.agents/skills`: instrumentation, evaluation setup, datasets, diagnosis, guardrails and notebook execution.
What does the diagnosis skill do?
It analyzes low scores and gives concrete recommendations for improving the app.
How does a guardrail differ from an evaluation?
An evaluation is scored after the answer, while `trulens-blocking-guardrails` sets a feedback function as a blocking check at runtime.
Is an external provider needed?
Usually yes for feedback functions: connect OpenAI, Gemini, Bedrock, LiteLLM and others through separate packages.
Discussion
No questions or reviews yet.
Related
SkillsNine skills from the AI Evals course authors: an agent reviews traces, finds failure modes and helps build evals for your product.
A skill that teaches Claude Code and Cursor to wire in Tracely tracing and turn failing agent runs into CI tests
Skill set for evaluating an AI product: rubric, dataset, failure codes, code checks and LLM judges with calibration.
Skill for browser testing of admin panels: runs cases from Excel and collects a report with failure evidence.
A skill and MCP server for the Rhesis platform: the agent designs, runs and analyzes test suites for your own AI agent
A CLI and MCP for testing agent skills: repeatable YAML evals, report comparison, regression detection and skilled-improve
README in short
The README describes TruLens as a tool for evaluating and tracking LLM and agent experiments. Installation goes through `pip install trulens` and provider packages.