HAMS

HAMS (Humanoid Agent Modular Stack) for the Correll Lab H1 robot: MuJoCo, ROS 2, and Isaac Sim running in separate containers and sharing a CycloneDDS ROS domain.

Layout

• docker/ — Dockerfiles, docker-compose.yml, and the build/run scripts.
• core_ws/ — ROS 2 workspace (bringup, IK, perception, safety). Submodules.
• h1_robocasa/ — RoboCasa/MuJoCo simulator entry point and bridges.
• CL_Assets/ — URDF, MuJoCo XML, and Isaac USD assets.

Prerequisites

• Docker (with Compose v2) and the NVIDIA Container Toolkit.

• Git LFS (git lfs install) — required to fetch the large binary assets (URDF meshes, MuJoCo XML, Isaac USD) tracked via LFS.

• git submodule update --init --recursive to populate core_ws/src.

• Copy docker/.env.example to docker/.env and fill in your GEMINI_API_KEY and ROS_DOMAIN_ID (see Configuration).

• SAM3 weights (sam3.pt) — needed by the perception/sam_server pipeline. They are not tracked in git; download them from the gated facebook/sam3 repo and place the file at core_ws/src/model_server/weights/sam3.pt:

  huggingface-cli login                         # one-time; accept the facebook/sam3 license
  cd core_ws/src/model_server/weights
  huggingface-cli download facebook/sam3 --local-dir .cache
  mv .cache/sam3.pt sam3.pt                      # name it exactly sam3.pt

  See core_ws/src/model_server/weights/README.md for details and the auto-download fallback.

A few things worth knowing before you run anything:

• The build/run scripts can be invoked from any directory — they resolve their own location, so docker/scripts/docker_run.sh robocasa works just as well from /tmp as from the repo root.
• ROS_DOMAIN_ID is read from docker/.env and forwarded into the RoboCasa and ROS containers. If it is unset or 0, it is normalized to 1 (domain 0 is reserved for the real robot).
• Isaac currently overrides this and pins its DDS bridge to channel 1 — see the Isaac section below.

Configuration

Runtime settings live in docker/.env (git-ignored — it holds your API key). Copy the template once and edit it; you never need to export these variables in your shell:

cp docker/.env.example docker/.env
# then edit docker/.env:
#   GEMINI_API_KEY=...   # https://aistudio.google.com/apikey
#   ROS_DOMAIN_ID=1      # any non-zero value; 0 is reserved for the real robot

Both docker compose and docker/scripts/docker_run.sh load docker/.env automatically, so every container and every terminal sees the same values. ROS_DOMAIN_ID is passed to all containers; GEMINI_API_KEY is passed only to the ros container (the vision pipeline's Gemini backbone and h12_skills need it). Because the run scripts source the file, docker/.env takes precedence over any value left exported in your shell.

Build the containers

docker/scripts/docker_build.sh             # all three
docker/scripts/docker_build.sh robocasa ros  # subset
docker/scripts/docker_build.sh isaac       # isaac only

The RoboCasa and ROS images both inherit from hams_base, which is built first automatically when either profile is selected. Isaac is self-contained and does not use the base.

Run the RoboCasa container

docker/scripts/docker_run.sh robocasa            # windowed viewer
docker/scripts/docker_run.sh robocasa --headless # no DISPLAY / SSH / CI
docker/scripts/docker_run.sh robocasa bash       # drop to a shell instead

# to use a custom ROS domain (e.g. several devs on one network),
# set ROS_DOMAIN_ID in docker/.env (see Configuration above)

Once it's up, RoboCasa publishes rt/lowstate over CycloneDDS plus /head/color/image_raw, /head/depth/image_raw, /head/color/camera_info, /lidar/points, and /tf on the chosen ROS domain (default 1).

Run the ROS container and bringup

The ROS launcher only builds the workspace and drops to a shell, so bringup is a manual step. ROS_DOMAIN_ID (all containers) and GEMINI_API_KEY (the ros container, for the vision pipeline's Gemini backbone and h12_skills) both come from docker/.env, so there is nothing to export — just open two terminals:

# terminal A — start RoboCasa first so /clock is publishing
docker/scripts/docker_run.sh robocasa

# terminal B — ROS workspace shell (auto-builds core_ws on first run)
docker/scripts/docker_run.sh ros

# inside the ROS container
ros2 launch h1_bringup h1_sim_bringup.launch.py

Bringup starts joint_state_publisher, robot_state_publisher, the frame_task_server IK solver, the safety_node, and rviz2.

Isaac container — WIP / TODO

The Isaac profile builds and runs the same way as the others:

docker/scripts/docker_build.sh isaac
docker/scripts/docker_run.sh  isaac

The launcher is docker/scripts/launch_isaac.sh. Task selection, asset paths, and the OmniGraph DDS bridge are not yet documented here. Note that the bridge currently unsets ROS_DOMAIN_ID and pins itself to channel 1 regardless of the host setting — bridging to a non-default domain is a known gap.

Troubleshooting

• X11 / GUI: run xhost +local:docker once per session if rviz, the MuJoCo viewer, or the slider GUI fail to open.
• Talking to the sim from the host (ros2 topic list, standalone rviz2) bypasses the run scripts, so it won't pick up docker/.env on its own — load it into your shell first: set -a; source docker/.env; set +a.
• For a clean rebuild of the message workspace, wipe container_cache/msgs_ws/ on the host before relaunching.

Working example — open the fridge

End-to-end run of the fridge-opening demo across three terminals. All three share ROS_DOMAIN_ID from docker/.env (terminals A and B via the run scripts, terminal C via the already-running container), so there's nothing to export.

# terminal A — RoboCasa (start first so /clock is publishing)
./docker/scripts/docker_run.sh robocasa --task OpenFridge --layout 1 --style 1 --seed 42

# terminal B — ROS workspace shell, then launch bringup
docker/scripts/docker_run.sh ros
ros2 launch h1_bringup h1_sim_bringup.launch.py

# terminal C — exec into the running ROS container and run the demo
docker exec -it hams_ros bash
source /opt/ros/humble/setup.bash
source /home/code/core_ws/install/setup.bash
ros2 action send_goal /skill/grasp custom_ros_messages/action/SkillGrasp \
  "{target_object: 'vertical fridge handle', arm: 'right', timeout: {sec: 60, nanosec: 0}}" \
  --feedback