SkillFoxx

Habitat-GS skill

Two Habitat-GS skills teach an agent to pilot a robot in 3D Gaussian Splatting scenes through MCP and to train navigation policies

  • 341 ★ on the repository
  • Last commit
Skill

Extended access

Why this level

  • The autonomous navigation loop starts a background process with its own LLM conversation that spends tokens
  • Policy training occupies the GPU and disk for a long time
All reasons and checks

zju3dv/habitat-gsAuthor:ZJU3DV

Install

Manual install

tools/manage_habitat_gs_skill.sh install --workspace /path/to/agent/workspace

Run it in the cloned repository after building the simulator. Symlink mode is enabled with --mode symlink and is not safe for every way of running the agent.

Access and checks

How we review

Extended accessThe tool runs code or makes network calls and only reads external data.

  • The autonomous navigation loop starts a background process with its own LLM conversation that spends tokens
  • Policy training occupies the GPU and disk for a long time

This is third-party code. Review the repository files before installing.

What it does

Habitat-GS is a navigation simulator for embodied AI with photorealistic scenes in 3D Gaussian Splatting format, an extension of Habitat-Sim (an ECCV 2026 paper). The repository holds two skills: habitat-gs-control and habitat-gs-train. habitat-gs-control teaches an agent to pilot a robot in a live session through MCP tools: load a scene with hab_init, look around with hab_look and hab_panorama, move with hab_forward and hab_turn, plan a route with hab_find_path, start an autonomous navigation loop with hab_nav_loop_start and save video with hab_export_video. It supports pointnav, objectnav, imagenav, eqa and instruction-following tasks. habitat-gs-train covers the full path of generating, training and evaluating PointNav, ImageNav and ObjectNav policies as well as the VLN models StreamVLN and Uni-NaVid. The HabitatAgent bridge with 16 tools starts with python tools/habitat_agent.py --mcp and works with Claude Code, Codex and OpenClaw. The script tools/manage_habitat_gs_skill.sh puts the skills into an agent workspace, and its default mode is the safe one: files are copied and symlinks appear only when you ask.

Who it is for. Navigation and embodied AI researchers who run the simulator on a GPU and hand robot control and experiments to Claude Code or Codex.

Good fit when

  • You want to steer a robot in a scene with words, for example "find the kitchen door"
  • PointNav or ObjectNav has to be trained on Gaussian Splatting scenes with a ready pipeline
  • A robot's walk through a scene must be recorded as video
  • The VLN models StreamVLN and Uni-NaVid are to be trained and evaluated in the same scenes

Not a fit when

  • The machine has no CUDA GPU: the recommended build enables CUDA and PyTorch is installed for cu121
  • You need a simulator without compilation: installation runs from source with submodules and cmake
  • The scene needs object physics: the simulator is built with Bullet for that and it is off by default

Example request

Load a scene with hab_init, find the kitchen door with the autonomous navigation loop and save a video of the walk

Limitations

  • The skills are useless without the simulator itself, which needs conda, Python 3.12, cmake 3.27, PyTorch for CUDA and downloaded Gaussian Splatting scenes.
  • habitat-gs-control forbids the agent from calling a shell: the session can be driven only through the hab_* MCP tools.
  • Skills in an agent workspace get their own .env file, separate from the main project's .env.
  • GitHub did not detect a license for the repository (NOASSERTION), so check the terms of use in the repository files.

How to disable. Remove the skills with tools/manage_habitat_gs_skill.sh uninstall --workspace /path/to/workspace. The same script with the status argument shows state. The MCP bridge stops when you end the tools/habitat_agent.py process.

FAQ

How do I install the Habitat-GS skills?

Clone the repository and run tools/manage_habitat_gs_skill.sh install --workspace /path/to/agent/workspace. A single skill installs with --skill habitat-gs-control or --skill habitat-gs-train.

How do habitat-gs-control and habitat-gs-train differ?

habitat-gs-control pilots a robot in a live session through MCP, while habitat-gs-train walks through data generation, training and evaluation of navigation policies. Training does not need session control.

Which agents work with the HabitatAgent bridge?

The author names Claude Code, Codex and OpenClaw. The bridge starts with python tools/habitat_agent.py --mcp and exposes 16 MCP tools.

Which navigation tasks does habitat-gs-control support?

The skill handles pointnav, objectnav, imagenav, eqa and instruction-following tasks through the hab_nav_loop_start tool.

Does Habitat-GS need a GPU?

The recommended build has CUDA on and PyTorch is installed for cu121, so a CUDA-capable NVIDIA GPU is needed.

Discussion

No questions or reviews yet.

Skills

A skill for Claude Code and Gemini CLI that runs LLM fine-tuning from dataset to GGUF export, on NVIDIA GPUs and Apple Silicon

SkillExtended access283

A library of 31 manipulation skills and 8 tool bundles for the GaP robotics engine in the Agent Skills format.

PluginBroad access64

A set of eight skills for AI infrastructure: TileLang GPU kernels, Megatron memory, SLIME training and diagrams.

SkillMinimal access158

90 specialist skills and the Nexus orchestrator for Claude Code, Codex and Antigravity: bugs, tests, security, review, UX

SkillExtended access99

A catalog of nearly two hundred skills: code delivery, review, debugging, AI governance, product work and agent coordination.

SkillMinimal access175

A meta-skill that splits a task into stages, picks a profile and runs multi-agent work in Codex and Claude Code.

SkillExtended access59

README in short

The README covers installing the simulator in detail, downloading Gaussian Splatting scenes, running the interactive viewer and avatars. Then come HabitatAgent, training pipelines for PointNav, ImageNav, ObjectNav and VLN, the skills section and the paper citation.

README badge

Original SKILL.md

---
name: habitat-gs-control
description: Interactively pilot a robot in habitat-gs 3D Gaussian Splatting indoor scenes via MCP tools — load a scene, move, observe with RGB/depth, run autonomous nav loops, and export video. Use for hands-on control of a live sim session, NOT for training or evaluating a navigation policy (use the habitat-gs-train skill for that).
---

# habitat-gs-control

Control a robot navigating through photo-realistic 3D indoor environments. You have MCP tools for scene loading, movement, observation, and autonomous navigation.

**IMPORTANT**: Always use MCP tools (`hab_init`, `hab_look`, `hab_forward`, etc.) directly. Do NOT use `exec` or shell commands — the MCP tools are the only supported interface.

## What you can do

- **Observe**: `hab_look` (current view), `hab_panorama` (4-direction scan with depth), `hab_depth_analyze` (obstacle distances)
- **Move**: `hab_forward` (walk forward), `hab_turn` (rotate left/right)
- **Navigate**: `hab_navigate` (auto-walk to coordinates), `hab_find_path` (plan route), `hab_topdown` (overhead map)
- **Long tasks**: `hab_nav_loop_start` (launch autonomous navigation sub-agent), `hab_nav_loop_status` (check progress), `hab_nav_loop_stop`
- **Session**: `hab_init` (load scene), `hab_close` (end session), `hab_export_video` (replay video)

## How to use

1. **Always init first**: call `hab_init` to load a scene before anything else.
2. **Look before you move**: use `hab_look` or `hab_panorama` to see the environment, then decide where to go.
3. **For simple tasks** (move forward, look around): use tools directly.
4. **For complex navigation** (find an object, reach a goal, follow instructions): use `hab_nav_loop_start` with a goal description. It launches an autonomous sub-agent that navigates independently. Check progress with `hab_nav_loop_status` only when asked.
5. **Export video** when the task is done: `hab_export_video` encodes all captured frames into an mp4.

## Navigation modes

The scene may run in two modes (auto-detected):

| | **navmesh** | **mapless** |
|---|---|---|
| Map tools (`hab_navigate`, `hab_find_path`, `hab_topdown`) | available | not available |
| Coordinates visible | yes | no — only `euclidean_distance_to_goal` and `goal_direction_deg` |
| How to navigate | use `hab_navigate` for efficient multi-step movement | use `hab_forward`/`hab_turn` with depth checks |

## Task types for nav_loop

| Type | What it does |
|---|---|
| `pointnav` | Navigate to a target coordinate |
| `objectnav` | Find a specific object in the scene |
| `imagenav` | Find the viewpoint matching a reference image |
| `eqa` | Explore and answer a question about the environment |

Report a problem