Using Visualizer Tiled Cameras

Using Visualizer Tiled Cameras#

For general visualizer documentation, see Visualization.

The visualizer tiled camera view is a live monitoring and debugging tool. It opens a non-interactive panel in the Kit or Newton visualizer and displays tiled camera views across all selected environments. They can stream observation camera data or generate cameras that follow the robots.

This guide is accompanied by the run_tiled_camera_visualizer.py script in the IsaacLab/scripts/tutorials/07_visualizers directory.

Running this script demonstrates two ways to use tiled cameras:

  • configured tiled cameras pointed at and following moving Anymal-D robots shown in the Kit visualizer

  • streaming from existing wrist-mounted robot cameras shown in the Newton visualizer

Note: Visualizer tiled cameras are currently supported only in the Kit and Newton visualizers. Either visualizer can be used to run either example.

Code for run_tiled_camera_visualizer.py
  1# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
  2# All rights reserved.
  3#
  4# SPDX-License-Identifier: BSD-3-Clause
  5
  6"""
  7This script demonstrates the visualizer tiled camera panel.
  8
  9.. code-block:: bash
 10
 11    # Kit visualizer tiled camera panel
 12    ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
 13        --enable_cameras --task Isaac-Velocity-Rough-Anymal-D-v0 --num_envs 256 --viz kit
 14
 15    # Newton visualizer tiled camera panel
 16    ./isaaclab.sh -p scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py \
 17        --task Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0 --num_envs 25 --viz newton
 18
 19"""
 20
 21from __future__ import annotations
 22
 23import argparse
 24import contextlib
 25import sys
 26
 27import gymnasium as gym
 28import torch
 29
 30import isaaclab_tasks  # noqa: F401
 31
 32with contextlib.suppress(ImportError):
 33    import isaaclab_tasks_experimental  # noqa: F401
 34from isaaclab_tasks.utils import (
 35    add_launcher_args,
 36    launch_simulation,
 37    resolve_task_config,
 38    setup_preset_cli,
 39)
 40
 41KIT_DEFAULT_TASK = "Isaac-Velocity-Rough-Anymal-D-v0"
 42NEWTON_DEFAULT_TASK = "Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0"
 43SUPPORTED_TILED_VISUALIZERS = {"kit", "newton"}
 44UNSUPPORTED_TILED_VISUALIZERS = {"rerun", "viser"}
 45
 46
 47def _requested_visualizers(args_cli: argparse.Namespace) -> list[str]:
 48    """Return requested visualizers, defaulting to Kit for this tutorial."""
 49    visualizers = args_cli.visualizer or ["kit"]
 50    visualizers = [str(visualizer).lower() for visualizer in visualizers]
 51
 52    if "none" in visualizers:
 53        raise ValueError("This demo requires a tiled-camera visualizer. Use '--viz kit' or '--viz newton'.")
 54    unsupported = sorted(set(visualizers) & UNSUPPORTED_TILED_VISUALIZERS)
 55    if unsupported:
 56        raise ValueError(
 57            "The visualizer tiled camera panel is only implemented for Kit and Newton. "
 58            f"Unsupported selection: {unsupported}."
 59        )
 60    unknown = sorted(set(visualizers) - SUPPORTED_TILED_VISUALIZERS)
 61    if unknown:
 62        raise ValueError(f"Unknown visualizer selection for this demo: {unknown}.")
 63    return visualizers
 64
 65
 66def _make_kit_visualizer_cfg(args_cli: argparse.Namespace):
 67    """Create generated tiled cameras that follow each Anymal-D base."""
 68    from isaaclab_visualizers.kit import KitVisualizerCfg
 69
 70    visualizer_cfg = KitVisualizerCfg()
 71    visualizer_cfg.tiled_cam_view = True
 72    visualizer_cfg.tiled_cam_num = 36
 73    visualizer_cfg.tiled_cam_prim_path = None
 74    # Here is an alternative eye position for a top down view
 75    # visualizer_cfg.tiled_cam_eye = (0.0, 0.0, 5.0)
 76    visualizer_cfg.tiled_cam_eye = (3.0, 3.0, 3.0)
 77    visualizer_cfg.tiled_cam_target_prim_path = "/World/envs/*/Robot/base"
 78    return visualizer_cfg
 79
 80
 81def _make_newton_visualizer_cfg():
 82    """Stream existing Galbot wrist cameras into the Newton tiled camera panel."""
 83    from isaaclab_visualizers.newton import NewtonVisualizerCfg
 84
 85    visualizer_cfg = NewtonVisualizerCfg()
 86    visualizer_cfg.tiled_cam_view = True
 87    visualizer_cfg.tiled_cam_num = 12
 88    # Here are other robot mounted camera options for this environment
 89    # visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/left_arm_camera_sim_view_frame/left_camera"
 90    # visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/right_arm_camera_sim_view_frame/right_camera"
 91    visualizer_cfg.tiled_cam_prim_path = "/World/envs/env_.*/Robot/head_camera_sim_view_frame/head_camera"
 92    visualizer_cfg.tiled_cam_eye = None
 93    visualizer_cfg.tiled_cam_target_prim_path = None
 94    return visualizer_cfg
 95
 96
 97def _configure_visualizers(env_cfg, args_cli: argparse.Namespace) -> None:
 98    """Attach tiled camera visualizer configs to the environment simulation config."""
 99    visualizers = _requested_visualizers(args_cli)
100    args_cli.visualizer = visualizers
101    env_cfg.sim.visualizer_cfgs = [
102        _make_kit_visualizer_cfg(args_cli) if visualizer == "kit" else _make_newton_visualizer_cfg()
103        for visualizer in visualizers
104    ]
105
106
107def _resolve_task(args_cli: argparse.Namespace) -> str:
108    """Resolve the task for the selected visualizer."""
109    if args_cli.task is not None:
110        return args_cli.task
111    if "newton" in _requested_visualizers(args_cli):
112        return NEWTON_DEFAULT_TASK
113    return KIT_DEFAULT_TASK
114
115
116# add argparse arguments
117parser = argparse.ArgumentParser(description="Showcase the Kit/Newton visualizer tiled camera panel.")
118parser.add_argument("--num_envs", type=int, default=None, help="Number of environments to simulate.")
119parser.add_argument("--task", type=str, default=None, help="Name of the task.")
120# append AppLauncher cli args
121add_launcher_args(parser)
122args_cli, hydra_args = setup_preset_cli(parser)
123args_cli.task = _resolve_task(args_cli)
124sys.argv = [sys.argv[0]] + hydra_args
125
126
127def main():
128    """Run a random-action environment with a tiled camera visualizer."""
129    # parse configuration via Hydra (supports preset selection, e.g. presets=newton_mjwarp)
130    env_cfg, _ = resolve_task_config(args_cli.task, "")
131    _configure_visualizers(env_cfg, args_cli)
132
133    with launch_simulation(env_cfg, args_cli):
134        # override with CLI arguments
135        env_cfg.scene.num_envs = args_cli.num_envs if args_cli.num_envs is not None else env_cfg.scene.num_envs
136        env_cfg.sim.device = args_cli.device if args_cli.device is not None else env_cfg.sim.device
137
138        # create environment
139        env = gym.make(args_cli.task, cfg=env_cfg)
140
141        # print info (this is vectorized environment)
142        print(f"[INFO]: Gym observation space: {env.observation_space}")
143        print(f"[INFO]: Gym action space: {env.action_space}")
144        env.reset()
145
146        # keep stepping until all visualizer windows have been closed
147        sim = env.unwrapped.sim
148        if not sim.visualizers:
149            print("[WARN]: No visualizers found. Exiting.")
150            env.close()
151            return
152
153        while True:
154            if sim.visualizers and not any(v.is_running() and not v.is_closed for v in sim.visualizers):
155                break
156            with torch.inference_mode():
157                actions = 2 * torch.rand(env.action_space.shape, device=env.unwrapped.device) - 1
158                env.step(actions)
159
160        env.close()
161
162
163if __name__ == "__main__":
164    main()

Example One: Following Anymal-D Robots#

The Kit Visualizer shows the tiled camera view in a separate tab inside the main Viewport window. The highlighted tab area in the figures below shows where to toggle between the interactive viewport and the visualizer tiled camera view.

Kit visualizer interactive viewport for Anymal-D robots

Kit visualizer showing the default interactive viewport.#

Kit visualizer tiled camera view for Anymal-D robots

Kit visualizer showing the tiled camera view generated for selected Anymal-D robots.#

Note, you can also display the main visualizer camera and the tiled camera view side by side for dual monitoring.

To run the tutorial with the args for this example, use:

python scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py --enable_cameras --task Isaac-Velocity-Rough-Anymal-D-v0 --num_envs 256 --viz kit

Within the script, you’ll find the KitVisualizerCfg configuration used to generate this example. You can use this config as a template for your own use cases.

In this example, a set of cameras is created to point toward each robot’s base prim and follow its motion. The camera’s position, relative to the prim, is set by the tiled_cam_eye field of KitVisualizerCfg. For this demo, the camera is offset by (3.0, 3.0, 3.0) from each robot base. If you change tiled_cam_eye (for example, to (0, 0, 5)), the panel will show a top-down view instead.

In this example, there are 256 total environments, and we randomly sample 36 to stream to the tiled camera view.

Also note that the Kit visualizer tiled camera view requires passing the --enable_cameras CLI arg.

Example Two: Streaming from Robot-Mounted Cameras#

The Newton visualizer provides a tiled camera view in a lightweight OpenGL window. Use the highlighted Tiled Camera View dropdown in the left-hand sidebar to show or hide the tiled camera panel.

Newton visualizer interactive view for the Galbot cube stacking environment

Newton visualizer showing the default interactive viewport.#

Newton visualizer tiled camera view for Galbot wrist cameras

Newton visualizer showing the selected Galbot head-camera feeds in the tiled camera panel.#

In this example, we use the Galbot cube stacking environment, which comes with built-in wrist-mounted cameras. This setup provides an egocentric view of the gripper, table, and cubes in each selected environment.

To launch this example, run:

python scripts/tutorials/07_visualizers/run_tiled_camera_visualizer.py --task Isaac-Stack-Cube-Galbot-Left-Arm-Gripper-Visuomotor-v0 --num_envs 25 --viz newton

Within the script, the NewtonVisualizerCfg is configured to stream images from the existing camera sensor located at /World/envs/env_.*/Robot/head_camera_sim_view_frame/head_camera. This path points to the head camera, but you can edit the tiled_cam_prim_path field of NewtonVisualizerCfg in the script to show a different existing camera if needed.

In this demo, 25 environments are simulated, and 12 camera feeds are shown in the tiled panel by default.

Configuration notes#

To customize tiled camera behavior, edit the highlighted VisualizerCfg fields in run_tiled_camera_visualizer.py:

  • For generated cameras, tiled_cam_target_prim_path chooses the followed prim and tiled_cam_eye sets the camera offset from that prim.

  • For existing scene cameras, tiled_cam_prim_path must match an Isaac Lab Camera sensor in the selected task.

  • tiled_cam_num controls how many environment tiles are shown.

Troubleshooting#

  • If a generated view fails with a missing prim error, check that tiled_cam_target_prim_path resolves in each selected environment. Common template forms include /World/envs/*/... and /World/envs/env_.*/....

  • If an existing-camera view reports that no Isaac Lab camera owns the prim, check that tiled_cam_prim_path matches a Camera sensor in the task.

  • If rerun or viser is selected, use --viz kit or --viz newton instead. The tiled camera panel is currently implemented for Kit and Newton.

  • If the view is too expensive, reduce tiled_cam_num, --num_envs, or the camera resolution. The visualizer caps the tiled panel at 100 tiles.

See also#