nnInteractive Agent plugin
3D image segmentation from a text request: an MCP server, a skill and a web demo on nnInteractive for Claude Code and Codex.
- 80 ★ on the repository
- Last commit
Broad access
Why this level
- The python tool runs arbitrary code as the user by default, and image renders go to the model provider.
Install
- Works in
Manual install
pip install "nninteractive-agent[local] @ git+https://github.com/MIC-DKFZ/nninteractive-agent"Then `nninteractive-agent setup claude` or `setup codex`.
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- The python tool runs arbitrary code as the user by default, and image renders go to the model provider.
This is third-party code. Review the repository files before installing.
What it does
The agent takes a request such as "segment every rib individually", draws prompts for the nnInteractive model (scribbles, boxes, points), checks the 3D result and corrects it like a human annotator. CT, MRI, microscopy, electron microscopy and plain photographs all work. The kit holds an `nninteractive` MCP server with tools such as `load_image`, `scribble`, `bbox`, `point`, `render3d`, `stats` and `save`, a `nninteractive-segmentation` skill with prompt rules and a correction playbook, and an optional web demo with 2D and 3D views and a chat. `nninteractive-agent setup claude` or `setup codex` registers the server, installs the skill and, for Claude Code, adds a verifier subagent.
Who it is for. Researchers and annotators of medical and microscopy images who need 3D segmentation without drawing masks by hand.
Good fit when
- On a CT scan every rib must be segmented individually and an instance map saved.
- On a microscopy stack all cell nuclei must be found and labelled.
- Segmentation is shown to colleagues with no agent installed through the web demo.
- Agent annotation quality is compared with the benchmark of 14 public images.
Not a fit when
- Images cannot be shared with the model provider, since the agent sees their renders.
- There is no NVIDIA GPU and compute time matters.
- The result goes to a clinic as a diagnostic conclusion.
Example request
Segment every rib on this CT as a separate object, check the result in 3D and save the instance map.Limitations
Python 3.10 or newer is required. Local inference targets an NVIDIA GPU with about 10 GB of memory; a CPU works, but every prompt takes long. The nnInteractive weights, about 400 MB, are downloaded to `~/.nninteractive`. The agent sees rendered slices, so only images you may show the model provider are suitable. The `python` tool runs code as you by default, and the `--python guarded` mode is not a sandbox. The web demo listens only on `127.0.0.1`, and when opened to a network anyone with the password gets access.
How to disable. Remove the `nninteractive` MCP entry from the Claude Code or Codex configuration and delete the skill from `~/.claude/skills` or `~/.codex/skills`. Downloaded weights sit in `~/.nninteractive`.
FAQ
What does the setup command do?
For local mode it checks PyTorch and the GPU and downloads weights. Then it registers the MCP server and installs the skill, plus a verifier subagent for Claude Code.
Can inference run on another machine?
Yes. Run `nninteractive-server` on a GPU machine and pass `--server URL` to `setup`. The agent then connects to the server instead of computing locally.
How do I limit the python tool?
The `--python` option takes `full`, `guarded` and `off`. In `guarded` only computing libraries are allowed and file and process access is blocked. It is no sandbox.
Which image formats are read?
NIfTI, NRRD, MHA, TIFF, PNG, JPG and NPY: `load_image` returns statistics, orientation and overview images.
How accurate is the annotation?
On the authors benchmark of 14 public images Claude Code and Codex reach a mean Dice of 0.81–0.85 in about four minutes per image. One click per target gives 0.71.
Discussion
No questions or reviews yet.
Related
PluginsMCP server that connects Claude Code and Codex to an open R or Python session in RStudio and Positron, where your data lives
SP-Mind is a Claude Agent SDK agent for spatial proteomics analysis: from image correction to cell type annotation.
A Claude Code agent for protein design: target research, antibody and nanobody generation, screening and candidate ranking.
A Claude Code plugin and MCP server: over 1000 scientific tools for biology, chemistry and medicine, data-check commands and routing skills
An MCP server of science agents: Claude Code analyzes spectra and images, plans experiments and prepares DFT calculations
Monte Carlo plugin for Claude Code and Cursor: table checks before queries, data incident triage and monitor creation.
README in short
The README covers features, installation with the `[local]` and `[web]` extras, Claude Code and Codex setup, tools, the web demo, server mode and benchmark results, with a separate section on `python` tool access and safeguards.