RigorPilot Skills
A skill set that runs a research repository from its README within bounds and keeps verifiable evidence, without rewriting the original README
- 486 ★ on the repository
- Last commit
- MIT
Broad access
Why this level
- The skill runs commands from someone else's README on your machine with no OS sandbox
- Approved commands have network access and can download data and weights
- Writing evidence creates files in the target repository
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add skills/rigorpilot-skillsDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The skill runs commands from someone else's README on your machine with no OS sandbox
- Approved commands have network access and can download data and weights
- Writing evidence creates files in the target repository
This is third-party code. Review the repository files before installing.
What it does
RigorPilot Skills reproduces results of research repositories from their README. The main skill, ai-research-reproduction, reads the documentation, picks the smallest documented target (model inference or evaluation) and runs it under time limits. Every failure is recorded as an assumption, deviation or blocker, and the meaning of the code is never bent to manufacture success. Work runs in three steps. Plan lists the candidate cmd-XX commands from the README, picks the safest one and returns a selection fingerprint without running anything. Run executes only the reviewed command bound to that fingerprint and writes evidence to a repro_outputs folder inside the target repository. Verify rechecks the saved data without rerunning the command. The evidence sits in ANNOTATED_README.md: the same README with a card inserted after each section, giving a verdict and links to commands and logs. Remove the insertions and you get the original file byte for byte. SUMMARY.md, COMMANDS.md, LOG.md, status.json and file hashes are stored alongside. The set also has skills for read-only repository analysis, preparing environment and weights, starting training, diagnosing before a patch, and three skills for exploratory research. These need explicit authorization and work on a separate branch.
Who it is for. For researchers and ML engineers who vet other people's paper repositories and want a reproducible report instead of "it ran on my machine".
Good fit when
- You need to check that the model evaluation promised in a README actually runs and get a log with exact commands
- A paper repository needs a careful training start with a time limit and recorded loss
- Before building your own experiment on someone else's code you want a report of assumptions, deviations and blockers
- You want to understand the structure of an unfamiliar research repository without writing to it
- You want a README annotated per section, showing what was checked, what was not run and what is blocked
Not a fit when
- You want a paper summary or a general research assistant: the skill works only with repository code
- You want to chase a number from the paper at any cost: the set does not change the protocol silently or tune the metric
- You do not trust the repository: the run is not isolated and commands reach your system
- You just need a one-off command run or environment setup: simpler skills exist for that
Example request
Use ai-research-reproduction: run the smallest README-documented evaluation, preserve the source and write evidence to repro_outputs, plus an annotated copy beside the original. Ask before large downloads or long trainingLimitations
Execution happens on your machine and there is no OS sandbox. Resource and budget limits are not hard quotas. You need Python 3.11 or newer, Git and Node.js 22.20 or newer for the skills installer. The optional standalone model runner supports Anthropic Messages and reviewed commands and has no recorded successful live acceptance yet: three attempts returned HTTP 502. The results in the README come from historical runs: selection-only and partial-training cases stay an incomplete evaluation, not a reproduced metric.
How to disable. Delete the skill folders such as ai-research-reproduction from the agent's skills directory; the repro_outputs folder and the RIGORPILOT_README.md copy in the target repository can be removed by hand. The skill does not modify the original README.
FAQ
How do I install RigorPilot Skills in Claude Code?
The main skill installs with npx skills add lllllllama/RigorPilot-Skills --skill ai-research-reproduction. To get the helper skills too, swap the flag for --all. The installer needs Node.js 22.20.0 or newer; on an EBADENGINE error check your version.
Does the skill change my README?
No. The original text stays as it was, and the annotated copy inserts cards after sections. Removing the insertions gives the original file byte for byte. The RIGORPILOT_README.md copy lands beside the original when the --source-adjacent-readme flag is set.
What do the card colors mean?
Green: selected checks passed. Blue: not executed. White: read only. Yellow: partial. Red: blocked. Purple: a human decision is needed. Green speaks about checks, and reproduction of the paper's results needs separate confirmation. Blue is not counted as a failure.
Is it safe to run someone else's repository with this skill?
There is no isolation guarantee: approved commands reach the host and network. The skill never selects setup or download commands as the target, and the plan changes only together with a new fingerprint. Run only trusted repositories.
What is in the repro_outputs folder?
ANNOTATED_README.md with per-section cards, SUMMARY.md with the outcome, COMMANDS.md with exact commands, LOG.md, status.json and a _runtime subfolder with process state and output. File hashes and sizes are recorded in evidence_manifest.json.
Discussion
No questions or reviews yet.
Related
SkillsFeynman — a terminal AI agent for scientific research: literature reviews, source checks, paper critique and comparing a paper with its code
Autoresearch for skills
autoresearch
A skill that improves another skill: runs it on test inputs, scores outputs with binary checks and mutates the prompt
ARIS
ARIS (Auto-Research-In-Sleep)
A skill set for autonomous machine learning research: literature review, idea discovery, experiments and paper writing with review by a separate model
A CLI skill manager for AI agents: install from repositories, translate between agent formats and deploy into their configs
A CLI and MCP server that turns doc sites, repos, PDFs and videos into skills and RAG-ready data
AI Research Skills
AI Research Skills Library
98 skills for ML research, from ideation and model training to inference, evaluation and paper writing
README in short
The README shows four historical runs on well-known repositories (micrograd, minGPT, PyTorch MNIST, nanoGPT Shakespeare) with annotated READMEs and honest caveats that selection without execution and partial training are not a reproduced metric. It then covers installation, the recommended first run, the limits, an index of nine skills and two helpers, the evidence folder contents and offline validation of the set.
README badge
Original SKILL.md
--- name: ai-research-reproduction description: Rigor Reproduce compatible skill slug for README-first deep learning repository reproduction. Use when the user wants an end-to-end, minimal-trustworthy flow that reads the repository first, selects the smallest documented inference or evaluation target, coordinates intake, setup, trusted execution, optional trusted training, optional repository analysis, and optional paper-gap resolution, enforces conservative patch rules, records evidence assumptions deviations and human decision points, and writes the standardized `repro_outputs/` bundle. compatibility: Requires Python 3.11+ and Git for bundled orchestration; target repositories may require additional reviewed dependencies, network access, or accelerators. --- # ai-research-reproduction ## Purpose Guide README-first deep learning reproduction toward the smallest trustworthy run with auditable evidence. Preserve documented meaning; record assumptions, deviations and blockers instead of changing semantics to manufacture success. Load specialized references only for a concrete uncertainty. ## Fast Path For a routine bounded run, keep the control path short: 1. Read the target README and only the target test/config/source needed to understand the documented command. 2. Run `scripts/orchestrate_repro.py --repo <repo> --plan-only --agent-output` with any explicit user timeout bound already supplied; review `command_candidates`, the selected `cmd-XX`, side-effect contract, selection fingerprint, and returned `reviewed_run_args`. 3. Run the selected candidate with `--run-selected --command-id <cmd-XX> --plan-fingerprint <fingerprint> --agent-output`. 4. Run `--verify-output --agent-output`; inspect detailed evidence files only when verification fails or the result is partial/blocked. 5. Deliver the bounded result and stop.