Autocontext skill
Autocontext runs an agent task through evaluation and keeps traces, reports and a playbook so the next run starts from accumulated lessons
- 1,314 ★ on the repository
- Last commit
- Apache-2.0
Broad access
Why this level
- Runs agents and models through the chosen provider and spends paid calls with keys from environment variables
- Writes traces, reports, playbooks and artifacts of every run to disk, and traces may contain task and answer contents
- Starts local MCP, HTTP and WebSocket servers; a non-loopback bind is protected only by a token
Install
- Works in
Manual install
uv tool install autocontext==0.19.1The Python version with the full feature set, the autoctx command.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Runs agents and models through the chosen provider and spends paid calls with keys from environment variables
- Writes traces, reports, playbooks and artifacts of every run to disk, and traces may contain task and answer contents
- Starts local MCP, HTTP and WebSocket servers; a non-loopback bind is protected only by a token
This is third-party code. Review the repository files before installing.
What it does
Autocontext is a harness for improving agent work. You state a goal in plain words, for example `autoctx solve "improve billing-dispute support replies" --iterations 3`, and it repeats the task over several iterations with evaluation of the result. It keeps useful findings and discards dead ends. It installs from PyPI with `uv tool install autocontext==0.19.1` or from npm with `bun add -g autoctx@0.19.0`. The PyPI package is `autocontext` and the executable is `autoctx`. Each run writes results to disk under `runs/<run_id>`: `trace.jsonl`, per generation `strategy.json`, `analysis.md` and `score.json`, a final `report.md` and artifacts. Accumulated lessons live in `knowledge/<scenario>`: `playbook.md`, `hints.md`, tools and context bundles. All of it can be inspected, diffed, replayed and exported. `mission` commands handle long multi-step goals, `investigate` does evidence-driven diagnosis, `simulate` models system behavior, and `train` in the Python version distills stable behavior into a cheaper local model. An agent can reach Autocontext in three ways. For Pi you install the `pi-autocontext` extension. For MCP clients, including Claude Code and Cursor, `autoctx serve mcp` starts a stdio server and `autoctx capabilities --json` lists its tools. For Hermes, `autoctx hermes export-skill` exports a skill. The repository also has the `autocontext-creator` skill for producing knowledge and `autocontext-consumer` for reading it and moving it between checkouts.
Who it is for. For developers and researchers who build agents and want to measure results across runs and accumulate verified practices between iterations.
Good fit when
- An agent has a repeating task and after each run you want an evaluation, a trace and a report rather than only final text
- You want to accumulate a playbook and hints for a task in plain files that can be read, diffed and moved between checkouts
- You collect data to fine-tune a cheaper local model on stable agent behavior (Python version, `train` command)
- An agent needs typed access to running, judging and inspecting runs through the `autoctx serve mcp` server
- You run on your own GPU through vLLM, Ollama or any OpenAI-compatible endpoint
Not a fit when
- You need a one-off code or text fix with no evaluation or repeats: the iteration loop adds model call costs
- The machine has no Node.js 22.19.0 or newer and you want the npm version: it requires it
- You want to improve the agent's own skills against test scenarios: upskill and skill-up in the catalog do that
- You need a server reachable from the network without a token: HTTP and WebSocket listen on loopback only, and a non-loopback bind requires a token of 32 or more characters
Example request
Install autocontext, run the task "improve billing-dispute support replies" for three iterations and show me the report with the scoreLimitations
- The project is large and moves fast: the Python (0.19.1), npm (0.19.0) and Pi extension (0.12.0) packages carry different versions. The Python version has the full feature set and the TypeScript version only part of it, for example `train` exists only in Python.
- Work goes through a model provider: Pi, Anthropic, OpenRouter, `claude-cli`, `codex`, OpenAI-compatible endpoints and others. Key variables are listed in `.env.example`, and the key owner pays for model calls.
- Executable skills and the model router in Python are off by default and apply only to a schema-migration pilot in Docker. Context changes apply only after matched trials confirm them.
- The author calls the reuse studies infrastructure evidence. They do not guarantee the stated costs and are not enough to authorize production use.
- Apache-2.0 license.
How to disable. Remove the CLI with `uv tool uninstall autocontext` or `bun remove -g autoctx`, and delete the entry with the `autoctx serve mcp` command from your MCP client settings. The `runs` and `knowledge` folders in the project hold results and are deleted by hand.
FAQ
How do I make an AI agent improve from run to run on one task?
Run `autoctx solve "goal" --iterations 3`. Autocontext evaluates each iteration, saves useful lessons in `knowledge/<scenario>/playbook.md` and hints in `hints.md`, and discards dead ends. The next run uses what was accumulated.
What stays on disk after a run?
A `runs/<run_id>` folder with `trace.jsonl`, generations (`strategy.json`, `analysis.md`, `score.json`), a `report.md` and artifacts, plus a `knowledge/<scenario>` folder with the playbook, hints, tools and context bundles. Everything is stored as files.
How do I connect Autocontext to Claude Code or Cursor through MCP?
Add a server with the command `autoctx` and arguments `serve mcp` to the client settings, or run `bunx autoctx serve mcp`. The server speaks stdio, and `autoctx capabilities --json` lists the tools with input schemas.
Which model provider is the quickest start?
Pi is the lowest-friction path because it uses the local agent auth. Set `AUTOCONTEXT_AGENT_PROVIDER=pi` and `AUTOCONTEXT_PI_COMMAND=pi`. Other setups can use `anthropic`, `openai-compatible`, `openrouter`, `claude-cli`, `codex` and `pi-rpc`.
Can Autocontext run on my own models?
Yes, a separate docs page covers the whole loop on vLLM, Ollama or any OpenAI-compatible endpoint. Local endpoints can declare `AUTOCONTEXT_PROVIDER_HOSTING=local` and a capability class of `fast`, `mid_tier` or `frontier`.
Is it safe to expose the control server to the network?
By default HTTP and WebSocket bind to loopback. A non-loopback bind fails until `AUTOCONTEXT_SERVER_TOKEN` holds a random value of at least 32 characters. Tokens in URLs are rejected, and before exposing it the author recommends reading the `persistent-host` security model.
Discussion
No questions or reviews yet.
Related
SkillsA skill with 25 commands to audit and improve the prompts, tools and agent designs of a project in Cursor, Claude Code and Codex
A plugin marketplace for agents: reflexion, code and PR review, git, TDD, DDD, specs and subagent-driven development
A multi-agent harness that verifies results by artifacts: stop hooks, an independent judge and an event log for Claude Code, Codex, Cursor and more
An MCP server that keeps large tool output out of the agent's context window and stores session memory in SQLite
A memory and coordination layer for coding agents: specs and issues live in git while agents work in parallel
An open coding agent for the terminal, browser and desktop that works with models from OpenAI and Claude to Ollama
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Repository stars | Main difference |
|---|---|---|---|
| Autocontext (this entry) | no | 1,314 | |
| upskill | yes | 761 | a strong model builds a skill from agent traces, a weak one uses it, and evaluation shows the gain |
| AutoHarness | no | 10,482 | a self-learning skill layer for Claude Code that derives skills from sessions, updates them and removes unused ones |
| Agent Apprenticeship | no | 1,603 | a CLI for training agents |
README in short
The repository root holds a Python package in `autocontext`, a TypeScript package in `ts`, a Pi extension, skills, examples, documentation, tests and knowledge-reuse comparisons. The front page covers installation for four surfaces, a 30-second run, three agent entry points, the layout of run results, a table of core commands and the security model of the control servers. It continues with detailed sections on context bundles, component attribution, kernel evolution and the 0.19.0 release notes.