data-master skill

---
name: data-master
description: "Analyze datasets with preserved raw inputs, explicit cleaning decisions, reproducible figures and PDF reports. Full statistics for one tabular file; descriptive time-series analysis; integrity profiling for text, image, audio, geographic and mixed inputs. Use for data cleaning, EDA, group comparisons, statistical reports, 跑数据 or 数据分析 with a traceable evidence trail."
allowed-tools: Bash(python *), Bash(py *), PowerShell(python *), PowerShell(py *), Bash(data-master *), PowerShell(data-master *), WebSearch, WebFetch
---

# data-master — universal data processing pipeline

Turn **any data** into a **cleaned, audited, Nature-grade report + charts** with one command. The agent reads the data, decides modality, picks methods, runs the pipeline, and writes evidence-backed outputs. The user never has to choose a statistical test or a chart type.

## Contract

Installed CLI: `data-master analyze <file> --out <dir>`; use
`data-master doctor --json` to diagnose the environment and
`data-master verify <dir> --json` to check saved artifact hashes.
`data-master demo --out <new-dir>` runs a synthetic example, and
`data-master skill install` installs this skill atomically from the npm or
Python distribution. Existing
`python scripts/run_pipeline.py` commands below remain supported; installing
the CLI alone does not register the skill until `skill install` is run.

- **Input**: `$ARGUMENTS` = a file, a folder, or a glob. Optional flags:
  - `--report nature` (default) | `standard` | `executive` — report template.
  - `--out <dir>` — overrides the output root.
  - `--answers <file>` — answers `answers.template.json` decisions by id; every id and action is validated, and changing the file invalidates the stale cleaning stages on resume. The file may also carry a `run` block declaring `unit_key` / `group` / `outcomes` / `design` / `journal` / `covariates` (validated against the profile; explicit CLI values always win). A decision may name a single `column` of a multi-column flag.
  - `--covariates <a,b>` — declare numeric covariates; the adjusted (ANCOVA) comparison becomes primary, unadjusted stays as cross-check, and the homogeneity-of-slopes assumption is tested and reported.
  - `--unit-key <column>`, `--group <column>`, and `--outcomes <a,b>` — all three must be explicitly declared before the analysis is labelled confirmatory. Automatic group/outcome selection is exploratory.
  - `--max-input-mb <n>` — override the 300 MB in-memory input gate (readers hold files whole, as text).
  - `--export-lockfile` — writes `lockfile.txt` with the exact dependency versions that ran.
- **Output** (all under `<dataset_dir>\outputs\`):
  - `cleaned/` — cleaned data + `cleaning_plan.json` + `cleaning_report.md` + reproducible `clean_*.py`
  - `analysis/` — `profile.json`, `stats.json`, `validation_report.md`, `chart_decisions.json`, `sensitivity.json`
  - `charts/` — `*.png` figures
  - `report.md` — main report (the citable source text)
  - `report.html` — self-contained shareable export of report.md (figures embedded; verification hashes it)
  - `report.pdf` — **primary deliverable**; rendered by a three-tier cascade
  - `executive_summary.md` — 1-page TL;DR
  - `report_build.json` — provenance for every number + PDF backend trail + self-audit + pipeline commit
  - `audit/final_audit.md` — 8-gate + journal-checklist verdict
  - `charts_rendered.json` — per-figure ink ratio and blank-page check
  - `answers.template.json` — the never-automatic decisions this dataset raised, fillable and fed back with `--answers`
  - `lockfile.txt` — (with `--export-lockfile`) exact dependency versions that ran
- **Invariant — additive only**: NEVER modify, move, or delete any raw input file. Re-runs may overwrite `outputs/`.
- **Full-auto with gates**: never block on a method choice; always pick the most defensible default and document. ONLY ask the user when the input itself is ambiguous.
- **Method provenance**: every AI judgment records `rule_source` + `alternative_considered` + `rejected_reason`. Every number cites the script that produced it.