Creating Visualization Markers

Creating Visualization Markers#

Visualization markers render debug geometry (frames, arrows, spheres, custom meshes) over the scene through markers.VisualizationMarkers. Markers are display-only: they carry no physics and do not affect the simulation.

For plain points, lines, and splines, Isaac Sim’s own isaacsim.util.debug_draw extension is lighter-weight. Use VisualizationMarkers when you need more complex shapes.

Supported on Kit, Newton GL, Rerun, and Viser; not yet on Newton RTX. See Visualization for enabling markers on a given visualizer.

Quick Start#

This guide is accompanied by the packaged markers example.

uv run --extra isaacsim isaaclab example markers

./isaaclab.sh example markers

Pass --visualizer newton_gl (or another supported backend) to switch visualizers; defaults to kit.

../../_images/markers.jpg

Every marker prototype from the example script, arranged in a grid. Each column rotates in place and periodically rolls forward to the next prototype.#

To stop, close the window or press Ctrl+C.

Code for markers.py
  1# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
  2# All rights reserved.
  3#
  4# SPDX-License-Identifier: BSD-3-Clause
  5
  6"""This script demonstrates different types of markers.
  7
  8.. code-block:: bash
  9
 10    # Usage with default PhysX physics and default kit visualizer.
 11    uvx --from 'isaaclab[isaacsim]' isaaclab example markers
 12
 13"""
 14
 15import argparse
 16from typing import TYPE_CHECKING
 17
 18from isaaclab.app import add_launcher_args, launch_simulation
 19from isaaclab.utils import instantiate
 20
 21parser = argparse.ArgumentParser(
 22    description="This script demonstrates different types of markers.",
 23    conflict_handler="resolve",
 24)
 25parser.add_argument("--physics", default="isaacsim_physx", choices=["isaacsim_physx"], help="Physics backend.")
 26parser.add_argument("--max_steps", type=int, default=-1, help="Stop after this many steps; negative runs forever.")
 27add_launcher_args(parser)
 28parser.set_defaults(visualizer=["kit"])
 29args_cli = parser.parse_args()
 30if args_cli.max_steps == 0 or args_cli.max_steps < -1:
 31    parser.error("--max_steps must be positive or -1.")
 32
 33import torch
 34
 35import isaaclab.sim as sim_utils
 36from isaaclab.markers.visualization_markers_cfg import VisualizationMarkersCfg
 37from isaaclab.physics import PhysicsCfg
 38from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR, ISAACLAB_NUCLEUS_DIR
 39from isaaclab.utils.math import quat_from_angle_axis
 40
 41if TYPE_CHECKING:
 42    from isaaclab.markers import VisualizationMarkers
 43
 44
 45def define_markers() -> "VisualizationMarkers":
 46    """Define markers with various different shapes."""
 47    marker_cfg = VisualizationMarkersCfg(
 48        prim_path="/Visuals/myMarkers",
 49        markers={
 50            "frame": sim_utils.UsdFileCfg(
 51                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/UIElements/frame_prim.usd",
 52                scale=(0.5, 0.5, 0.5),
 53            ),
 54            "arrow_x": sim_utils.UsdFileCfg(
 55                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/UIElements/arrow_x.usd",
 56                scale=(1.0, 0.5, 0.5),
 57                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 1.0)),
 58            ),
 59            "cube": sim_utils.CuboidCfg(
 60                size=(1.0, 1.0, 1.0),
 61                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.0, 0.0)),
 62            ),
 63            "sphere": sim_utils.SphereCfg(
 64                radius=0.5,
 65                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 0.0)),
 66            ),
 67            "cylinder": sim_utils.CylinderCfg(
 68                radius=0.5,
 69                height=1.0,
 70                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 0.0, 1.0)),
 71            ),
 72            "cone": sim_utils.ConeCfg(
 73                radius=0.5,
 74                height=1.0,
 75                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 1.0, 0.0)),
 76            ),
 77            "mesh": sim_utils.UsdFileCfg(
 78                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
 79                scale=(10.0, 10.0, 10.0),
 80            ),
 81            "mesh_recolored": sim_utils.UsdFileCfg(
 82                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
 83                scale=(10.0, 10.0, 10.0),
 84                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.25, 0.0)),
 85            ),
 86            "robot_mesh": sim_utils.UsdFileCfg(
 87                usd_path=f"{ISAACLAB_NUCLEUS_DIR}/Robots/ANYbotics/ANYmal-C/anymal_c.usd",
 88                scale=(2.0, 2.0, 2.0),
 89                visual_material=sim_utils.GlassMdlCfg(glass_color=(0.0, 0.1, 0.0)),
 90            ),
 91        },
 92    )
 93    return instantiate(marker_cfg)
 94
 95
 96def main():
 97    """Main function."""
 98    with launch_simulation(cfg=PhysicsCfg(), launcher_args=args_cli) as physics_cfg:
 99        sim_cfg = sim_utils.SimulationCfg(dt=0.01, device=args_cli.device, physics=physics_cfg)
100        sim = sim_utils.SimulationContext(sim_cfg)
101        # Set main camera
102        sim.set_camera_view([0.0, 18.0, 12.0], [0.0, 3.0, 0.0])
103
104        # Spawn things into stage
105        # Lights
106        cfg = sim_utils.DomeLightCfg(intensity=3000.0, color=(0.75, 0.75, 0.75))
107        cfg.func("/World/Light", cfg)
108
109        # create markers
110        my_visualizer = define_markers()
111
112        # define a grid of positions where the markers should be placed
113        num_markers_per_type = 5
114        grid_spacing = 2.0
115        # Calculate the half-width and half-height
116        half_width = (num_markers_per_type - 1) / 2.0
117        half_height = (my_visualizer.num_prototypes - 1) / 2.0
118        # Create the x and y ranges centered around the origin
119        x_range = torch.arange(-half_width * grid_spacing, (half_width + 1) * grid_spacing, grid_spacing)
120        y_range = torch.arange(-half_height * grid_spacing, (half_height + 1) * grid_spacing, grid_spacing)
121        # Create the grid
122        x_grid, y_grid = torch.meshgrid(x_range, y_range, indexing="ij")
123        x_grid = x_grid.reshape(-1)
124        y_grid = y_grid.reshape(-1)
125        z_grid = torch.zeros_like(x_grid)
126        # marker locations
127        marker_locations = torch.stack([x_grid, y_grid, z_grid], dim=1)
128        marker_indices = torch.arange(my_visualizer.num_prototypes).repeat(num_markers_per_type)
129
130        sim.reset()
131        print("[INFO]: Setup complete...")
132
133        # Yaw angle
134        yaw = torch.zeros_like(marker_locations[:, 0])
135        step_count = 0
136        # Step while a visualizer window is still open (or none exist, e.g. headless); works for kit and newton.
137        while sim.is_running() and (args_cli.max_steps < 0 or step_count < args_cli.max_steps):
138            # rotate the markers around the z-axis for visualization
139            marker_orientations = quat_from_angle_axis(yaw, torch.tensor([0.0, 0.0, 1.0]))
140            # visualize
141            my_visualizer.visualize(marker_locations, marker_orientations, marker_indices=marker_indices)
142            # roll corresponding indices to show how marker prototype can be changed
143            if yaw[0].item() % (0.5 * torch.pi) < 0.01:
144                marker_indices = torch.roll(marker_indices, 1)
145            # perform step
146            sim.step()
147            step_count += 1
148            # increment yaw
149            yaw += 0.01
150
151
152if __name__ == "__main__":
153    main()

Configuring markers#

VisualizationMarkersCfg takes:

  • prim_path: prim path where the marker UsdGeom.PointInstancer is created.

  • markers: a dict of marker prototypes. The key names the prototype; the value is its spawn config (any SpawnerCfg, including USD file references).

Note

Physics properties on a marker prototype’s spawn config are stripped on creation, since markers are not simulated.

def define_markers() -> "VisualizationMarkers":
    """Define markers with various different shapes."""
    marker_cfg = VisualizationMarkersCfg(
        prim_path="/Visuals/myMarkers",
        markers={
            "frame": sim_utils.UsdFileCfg(
                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/UIElements/frame_prim.usd",
                scale=(0.5, 0.5, 0.5),
            ),
            "arrow_x": sim_utils.UsdFileCfg(
                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/UIElements/arrow_x.usd",
                scale=(1.0, 0.5, 0.5),
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 1.0)),
            ),
            "cube": sim_utils.CuboidCfg(
                size=(1.0, 1.0, 1.0),
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.0, 0.0)),
            ),
            "sphere": sim_utils.SphereCfg(
                radius=0.5,
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 1.0, 0.0)),
            ),
            "cylinder": sim_utils.CylinderCfg(
                radius=0.5,
                height=1.0,
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(0.0, 0.0, 1.0)),
            ),
            "cone": sim_utils.ConeCfg(
                radius=0.5,
                height=1.0,
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 1.0, 0.0)),
            ),
            "mesh": sim_utils.UsdFileCfg(
                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
                scale=(10.0, 10.0, 10.0),
            ),
            "mesh_recolored": sim_utils.UsdFileCfg(
                usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/DexCube/dex_cube_instanceable.usd",
                scale=(10.0, 10.0, 10.0),
                visual_material=sim_utils.PreviewSurfaceCfg(diffuse_color=(1.0, 0.25, 0.0)),
            ),
            "robot_mesh": sim_utils.UsdFileCfg(
                usd_path=f"{ISAACLAB_NUCLEUS_DIR}/Robots/ANYbotics/ANYmal-C/anymal_c.usd",
                scale=(2.0, 2.0, 2.0),
                visual_material=sim_utils.GlassMdlCfg(glass_color=(0.0, 0.1, 0.0)),
            ),
        },
    )
    return instantiate(marker_cfg)

Drawing markers#

visualize() sets marker poses and, optionally, which prototype each marker instance uses via marker_indices.

my_visualizer.visualize(marker_locations, marker_orientations, marker_indices=marker_indices)

Arguments left as None keep their previous value. Passing a different number of rows than the last call resizes the marker instance count. See visualize()’s docstring for the full argument list (translations, orientations, scales, marker_indices, environment_ids).

Markers in practice#

Markers are commonly used to debug per-environment state during training, such as commanded vs. current velocity, or contact events:

Velocity arrow marker on an AnymalD robot

Velocity command (green) and current velocity (blue) arrow markers on an AnymalD robot.

Joint arrow markers on a Franka arm and contact sensor markers on a cube

Joint arrow markers on a Franka arm, with contact sensor markers on a cube.

See also#

  • Visualization: enabling markers per visualizer, and other visualizer features

  • Recording Video: recording a marker-annotated scene to video