openreality-mcp
An MCP server that turns phone video into a 3D scene and lets an agent measure it, plan paths and export data for robots
- 656 ★ on the repository
- Last commit
- BSD-2-Clause
Broad access
Why this level
- Videos and recordings go to the Open Reality cloud service and are stored in your account there
- Server-side agents spend the account budget, and the login key sits on disk in a credentials file
- Through the hardware.json file the server can drive connected equipment, such as a robot arm
Install
- Works in
In your terminal, with SkillFoxx CLI
npx skillfoxx add mcp/openreality-mcpDetects the agents on your machine, shows the access level and pins the version.
Other ways to install
Access and checks
How we reviewBroad accessThe tool can change data, run commands or handle secrets.
- Videos and recordings go to the Open Reality cloud service and are stored in your account there
- Server-side agents spend the account budget, and the login key sits on disk in a credentials file
- Through the hardware.json file the server can drive connected equipment, such as a robot arm
This is third-party code. Review the repository files before installing.
What it does
openreality-mcp connects an agent to the Open Reality system, which builds a persistent 3D scene from a phone video. Through more than forty tools the agent uploads a video, a robot recording or a splat, waits for the reconstruction to finish, reads the scene card and asks which objects the scene holds and how large the room is. Measurements follow an honest-units rule. A length is called metres only after calibration with two points of known distance; until then the server returns relative units and the agent may not convert them. Beyond that you get distances and angles between points, path planning through free space to an object or point, and server-side scene agents that survey the scene, label objects and answer questions. For robot work the server exports a scene in the openreality, groot_lerobot_v2, isaac_usd and mujoco_mjcf formats. Heavy files are written to your disk, never into the model's context. The Claude Code plugin installs a skill with the working order together with the server. A separate `openreality-mcp simulator` command starts an offline simulator on fixture data, with no account and no GPU.
Who it is for. For developers and robotics and spatial-AI researchers who want to give an agent room scans and get measurements, paths and training data back.
Good fit when
- You need to turn a room video into a scene and learn the room's size and its list of objects
- You need to calibrate a scene with a known distance and measure an item or opening in metres
- You need a path through free space from the door to the desk, described in words
- You need training data for a robot in LeRobot, GR00T, Isaac Sim or MuJoCo formats from a real scan
- You want to rehearse the workflow with no account and no GPU on the built-in simulator
Not a fit when
- The video cannot go to the cloud: the reconstruction service keeps scenes in your account on Open Reality's side
- You need commercial reconstruction on your own server: the model's license terms do not allow it
- You need a ready photogrammetry package with its own interface: here everything goes through MCP tools
- You need metres without calibration: the server refuses to call relative units metres
Example request
Upload ~/Videos/kitchen.mp4 and reconstruct it, then tell me which objects are in it and how big the room isMCP
- Transport
- stdio
- Authentication
- key is optional
Environment variables
OPENREALITY_TOKENsecret- An ork_... key or sign-in token used instead of stored credentials
OPENREALITY_URL- Broker address: the hosted one by default, set it for a self-hosted server
OPENREALITY_LOGIN_URL- Browser sign-in hand-off page, https://open-reality.io by default
Limitations
The MCP server runs on your machine as a stdio process, while reconstruction, storage and the scene agents run on an Open Reality broker, so real use needs an account: `openreality-mcp login` opens a browser and stores a key in `~/.config/openreality/credentials.json`. Server-side agents spend a bounded account budget, and replaying a recorded run is free.
You can host the broker yourself: the server and core folders are in the repository and run on your own GPU box or your own Modal account. For such a server the VGGT-1B reconstruction model is fetched from its owners under CC BY-NC 4.0, which means non-commercial use only. Commercial work needs the hosted service or your own license. The server and core folders are published as mirrors of private repositories and are synced by hand. Simulator exports need a metric anchor on the scene, and MuJoCo collision covers only a floor plane and boxes around detected objects. Node 20 or newer is required.
How to disable. Run `claude mcp remove openreality`, or `codex mcp remove openreality` for Codex. List and revoke login keys with `openreality-mcp keys list` and `openreality-mcp keys revoke <key_id>`, and delete `~/.config/openreality/credentials.json` if you like.
FAQ
How do I connect openreality-mcp to Claude Code?
`claude mcp add openreality -- npx -y openreality-mcp serve` registers the server. The `--scope user` flag makes it available in every project. Then run `npx -y openreality-mcp login` once; sign-in opens in the browser.
Why does the server not call values metres?
Video reconstruction gives a relative scale. The server returns metres only after the scene is calibrated with two points of known distance, and the response says units: m. The agent gets refusals and uncertainty labels unedited and cannot pass a relative number off as metres.
Do I need a GPU?
Not for the Open Reality cloud. A self-hosted server needs your own GPU or a Modal account. The built-in simulator needs no GPU and no account, but reconstructs nothing.
Which formats can robot data be exported in?
openreality, groot_lerobot_v2, isaac_usd and mujoco_mjcf. The two simulator formats need a metric anchor on the scene first. Files are written to disk and the agent receives paths.
Can it drive real hardware?
Yes, through an optional hardware.json. For each device, such as an SO-101 robot arm, openreality-mcp starts its own MCP server as a child process. Device calls never reach the broker, and the agent sees the device's limits and refusals unedited.
Can I use the result in a commercial project?
The hosted service uses a commercially licensed model. A self-hosted server falls under the model's own license terms, so read them before launch.
Discussion
No questions or reviews yet.
Related
MCP serversBlender MCP
MCP for Blender
An MCP server and Blender addon: the agent creates and edits objects, materials and scenes, pulls assets and runs Python in Blender
Blender MCP: an agent in Cursor, Cline, Windsurf or Claude Desktop builds and edits 3D scenes in Blender from a description; the server installs with uvx
Screenplay films screen recordings from one prompt: the agent writes a scenario, the tool plays and edits it.
The Posecode MCP server teaches a model a text language for movement, checks joint angles and returns a link to a 3D animation.
MCP server of the 37Studio video studio: an agent manages brands, makes videos, posts to social networks and answers messages.
A local MCP server for editing and analyzing images: the agent crops frames, finds objects and reads text from photos
Comparison with similar tools
Cards that share the main search query, and how they differ.
| Tool | Official | Authentication | Repository stars | Main difference |
|---|---|---|---|---|
| openreality-mcp (this card) | yes | key is optional | 656 | |
| ROS MCP Server | no | not required | 1,525 | connects an agent to a ROS robot through rosbridge: sensors and commands, without scene reconstruction |
| Bagel: questions to robot and drone data | yes | not required | 407 | answers questions about robot, drone and IoT logs, but builds no scenes |
README in short
The repository description shows a Claude Code session demo beside a 3D panel, a feature table and a quick start for four clients. Sample requests follow, then a self-hosting section with a warning about the model license, an index of the mcp, server and core folders and a data-flow diagram. It ends with the projects the reconstruction builds on.