WikiSkill CLI
A CLI that runs your agent on tasks and revises its skills, keeping a change only when held-out tasks confirm the gain
- 285 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- Runs the agent with automatic approval of actions, for Codex the `--full-auto` flag, and the agent writes files inside the workspace
- Rollback uses `git reset --hard` inside the workspace, so unsaved work must not live there
- Every iteration calls a paid model on all tasks of the bench
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add cli/wikiskillDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Runs the agent with automatic approval of actions, for Codex the `--full-auto` flag, and the agent writes files inside the workspace
- Rollback uses `git reset --hard` inside the workspace, so unsaved work must not live there
- Every iteration calls a paid model on all tasks of the bench
This is third-party code. Review the repository files before installing.
What it does
WikiSkill is a Python implementation of a Google Research paper on skill evolution for agents. It runs your agent on training tasks, saves the execution traces and hands them to a maintainer agent. The maintainer distills the root causes and fixes it finds into a wiki of pattern pages. A proposer agent then reads the wiki and the traces and suggests a single change: create a skill or patch an existing one. The change is checked on held-out tasks. The score must strictly beat the best earlier one, otherwise a `git reset --hard` returns the skills to the previous set. The wiki survives a rollback, so knowledge accumulates while only confirmed edits reach the skills. The same agent plays both student and teacher. The commands `wikiskill init`, `status`, `evolve`, `run-task`, `compare` and `transfer` create a workspace, show its state, run the loop, replay one task, compare two runs and move skills between workspaces. The demo bench has 22 auto-graded tasks, and your own tasks are described in JSON. Backends: Hermes Agent as the main one, plus Claude Code, Codex and GitHub Copilot CLI.
Who it is for. For people who write skills for an agent and want to test edits on auto-graded tasks instead of gut feeling.
Good fit when
- You want the agent to refine its own skills and to accept only edits that raised the score on held-out tasks
- You have tasks that grade automatically: comparison with an expected file, a substring match, a JSON field or running the produced script
- You need to compare two runs with a paired test and learn whether a skill actually helped
- Skills grown on one agent have to move to another with `wikiskill transfer`
- You want evolution to run overnight on a schedule; the repository has a cron note for Hermes
Not a fit when
- Tasks cannot be graded automatically: the loop relies on a numeric score from the grader
- You need a ready set of vetted skills: the five skills in the repository come from run logs and none has passed the gate yet
- You have no time or budget for dozens of agent runs: one evolution step calls the agent on every task of the bench
- You want to change the agent's working profile on the fly: WikiSkill works in a separate isolated profile
Example request
Create a workspace with wikiskill init demo --backend claude, run wikiskill evolve --iters 3 and tell me which skills passed the gate and which were rolled backLimitations
You need Python 3.10 or newer and one supported agent that is already installed and signed in: `hermes`, `claude auth login`, `codex login` or `copilot login`. The model is billed at your agent's rate. The README quotes about 0.09 dollars per iteration on a weak free model through OpenRouter.
The live runs in the README have not yet produced an accepted skill: the gate rejected either neutral or harmful proposals, and in one run the proposer declined to change anything. The author states that accumulation needs a stronger model. One run was thrown out because of a bug in the framework itself, which was fixed and covered by a test.
PyPI releases before 0.1.5 shipped without the framework skills, so such workspaces must be recreated with `init`. OpenCode is planned but not supported.
How to disable. Remove the package with `pip uninstall wikiskill` and delete the `workspaces/` folder in the directory where you ran it. The program never touches your agent's own profile, since each workspace gets its own isolated one.
FAQ
What is WikiSkill and where does the idea come from?
It implements the WikiSkill paper, arXiv 2608.27454, written at Google Research. The idea is that an agent's experience accumulates in a persistent wiki while skills change through a four-step loop. The agent does tasks, a maintainer distills traces into the wiki, a proposer patches one skill, and a gate accepts the patch or rolls it back.
Why is the wiki not rolled back together with the skills?
Knowledge about failure causes is valuable even when a particular skill edit did not help. Rejected proposals are stored in full in `wiki/skill-impact.md` so the next proposer does not repeat them.
Which agents does WikiSkill work with?
The main backend is Hermes Agent. Others are Claude Code through `claude -p`, Codex through `codex exec --json --full-auto` and GitHub Copilot CLI through `copilot -p -s`. The backend is pinned with `--backend` when the workspace is created and can be switched in `evolve`.
Which tasks can be plugged in and how are they graded?
Tasks live in `tasks.json`: an id, a train or val split, the prompt, sandbox files and a grader. Graders are `exact`, `contains`, `json_field` and `code_stdout`, the last one running the produced script. A missing deliverable scores zero and never crashes the run.
How do I install the repository's skills in Hermes?
The repository doubles as a skill source for Hermes. Run `hermes skills tap add ashutoshsinghpr7/wikiskill`, then `hermes skills install ashutoshsinghpr7/wikiskill/skills/wikiskill-evolve`, one command per skill you want. The author notes that none of them has been accepted by the gate yet.
How do I check whether a skill helped?
The `wikiskill compare` command takes two workspaces and counts a win, loss or tie per task. At the end it prints a two-sided exact binomial p-value.
Discussion
No questions or reviews yet.
Related
CLI toolsA skill that designs, tests, evaluates and packages other agent skills, starting from architecture
A CLI skill manager for AI agents: install from repositories, translate between agent formats and deploy into their configs
Two skills that pick other skills for a task: the hosted one fetches them from an index and collects feedback, the local one searches your SKILL.md folders
Self-learning
self-learning-skills
A meta-skill that saves a hard-won working procedure as a new skill or rule, so the next session starts already knowing it
A Go terminal UI for agent skills: list, check, update and find new ones in a single window
Darwin Skill
darwin-skill
A skill that improves other skills: scores SKILL.md on nine criteria, edits, tests with prompts and reverts
Comparison with similar tools
Entries that share the main search query, and how they differ.
| Entry | Official | Repository stars | Main difference |
|---|---|---|---|
| WikiSkill (this entry) | no | 285 | |
| Hermes Agent | no | 258,249 | the agent itself, for which WikiSkill grows skills |
| Self-Improving Agent | no | 782 | an in-session self-improvement skill without held-out tasks or rollback |
README in short
The README opens with the four-step loop and an evolution diagram, then gives a quick start and a command table. It goes on to live results with gate rejections, design decisions, a backend table and the list of ready skills, and ends with a roadmap of finished items.