Pier CLI
A Harbor fork for evaluating coding agents in sandboxes: Claude Code, Codex and others run inside tasks with no internet.
- 231 ★ on the repository
- Last commit
- Apache-2.0
Extended access
Why this level
- Runs execute agent code in docker or modal sandboxes and spend paid model tokens.
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add cli/pierDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewExtended accessThe tool runs code or makes network calls and only reads external data.
- Runs execute agent code in docker or modal sandboxes and spend paid model tokens.
This is third-party code. Review the repository files before installing.
What it does
Pier reads the Harbor task format and runs coding agent trials against it in isolated environments. `pier run -p path/to/task --agent claude-code --env modal` runs a task, and `--n-tasks 10 --sample-seed 0` takes a deterministic random subset of a local dataset. It differs from Harbor in three ways. Agents that run inside the sandbox declare install scripts and a network allowlist, and the `docker` and `modal` environments honor them, so Claude Code and Codex work in tasks with `allow_internet = false`. Trajectories are written in augmented ATIF v1.7: one step per API turn, reasoning separated from agent messages, with `peak_context_tokens` and `llm_call_count` added. There is a chat-style trajectory viewer `pier view` and `pier critique run`, which reviews finished trials with a fresh agent in a fresh sandbox. Supported agents are `claude-code`, `codex`, `cursor-cli`, `gemini-cli`, `opencode`, `mini-swe-agent` and the service agents `nop` and `oracle`.
Who it is for. Researchers and engineers who build benchmarks for coding agents and compare them on tasks with no internet access.
Good fit when
- Claude Code has to be run on DeepSWE tasks in a closed sandbox with no network.
- A task set in Harbor format needs a ten-task sample with a fixed seed.
- Run trajectories are required in a strict format with context token counts.
- Finished trials should be reviewed by a separate agent in a clean environment.
Not a fit when
- You need Harbor registry datasets directly: Pier does not download them, they are fetched through Harbor.
- You need an agent for everyday work with no evaluation: Pier is built for trials.
- You need environments other than `docker` and `modal`: others are not described yet.
Example request
Run Claude Code on the datasets/swebenchpro task directory in the modal environment, ten random tasks with seed 0.Limitations
Today Pier reads local task directories in Harbor format, and datasets from the Harbor registry are first downloaded with Harbor itself using `harbor download`. The `docker` and `modal` environments are supported. Running agents needs their keys, for example `ANTHROPIC_API_KEY`, and these are paid model calls. The project is open and young, commands may change, and `pier run --help` gives the current list. The license is Apache-2.0, the last commit dates from October 2026.
How to disable. Remove the package with `uv tool uninstall datacurve-pier` or `pip uninstall datacurve-pier`. Run results sit in the `jobs/` directory and are deleted separately.
FAQ
How do I install Pier?
Run `uv tool install datacurve-pier` or `pip install datacurve-pier`. The package is published on PyPI.
How does Pier differ from Harbor?
It supports agents installed inside a sandbox with no internet, writes ATIF v1.7 trajectories and adds `pier view` and `pier critique run`.
Which agents are supported?
`claude-code`, `codex`, `cursor-cli`, `gemini-cli`, `opencode`, `mini-swe-agent`, plus the service agents `nop`, `oracle` and `antigravity-sdk`.
Where are the results stored?
In the `jobs/` directory by run hierarchy. They can be browsed with `pier view`.
How do I use a dataset from the Harbor registry?
Download it with Harbor itself, `harbor download swebenchpro -o datasets`, then pass the path through `pier run -p`.
Discussion
No questions or reviews yet.
Related
CLI toolsRuns Claude Code, Codex, Gemini and OpenCode on YAML tasks in a sandbox, scores the result and gates CI on it
Vercel CLI for testing coding agents on your framework: runs Claude Code, Codex and others and measures the success rate
A CLI that rates the maturity of an AI agent harness in a repository from L0 to L4: 36 file checks with no model calls or network.
Hooks for Claude Code and Codex CLI keep a ledger of agent actions and refuse a "done" claim until a check has passed.
An evaluation harness for coding agents: 26 compact tasks in 6 languages in Docker containers, weighted score and integrity checks
A local proxy and trace viewer for coding agents: shows system prompts, tool calls and token usage for Claude Code, Codex and other CLIs
README in short
The pier CLI, compatible with Harbor tasks: runs Claude Code, Codex and other agents in docker and modal sandboxes with ATIF v1.7 trajectories.