Recording video

Recording video#

This guide shows how to record mp4 clips from a visualizer or a scene sensor during a run.

Recorded clip of the Shadow Hand cube-reorientation task, recording the streaming view of the Rerun visualizer with 4 streaming view envs and 4 GT types

Each VideoRecorderCfg entry is independent: different sources write different files at their own cadence, with no limit on simultaneous recorders.

Quick Start#

Add a VideoRecorderCfg to env_cfg.video_recorders:

from isaaclab.envs.utils.video_recorder_cfg import VideoRecorderCfg

env_cfg.video_recorders = [
    VideoRecorderCfg(source="visualizer:kit", output_dir="videos/")
]

Or pass --video on the command line to record without editing the environment config. It records the first capture-capable visualizer selected by --viz. Without --viz, it uses the first configured visualizer (preferring one that supports capture), or creates a headless Kit visualizer when none is configured. To record several sources at once, list one VideoRecorderCfg per source in env_cfg.video_recorders; --video_length and --video_interval apply to every recorder:

uv run --extra isaacsim --extra video isaaclab train --rl_library rsl_rl \
    --task Isaac-Cartpole --video

See the –video sources for the command-line selection rules, and Sources and Clip control below for the VideoRecorderCfg options. To record from a different camera angle than the interactive view, see Recording from an independent camera angle.

Examples#

All three examples use the Shadow Hand cube-reorientation task, Isaac-Reorient-Cube-Shadow-Camera-Direct, which ships with a built-in tiled camera sensor. Examples 1 and 2 each demonstrate one recording source; Example 3 combines all four.

Code for run_video_recording.py
  1# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
  2# All rights reserved.
  3#
  4# SPDX-License-Identifier: BSD-3-Clause
  5
  6"""Tutorial: recording video from visualizers and scene sensors.
  7
  8This script demonstrates three progressively richer recording configurations,
  9all using the Shadow Hand cube-reorientation task
 10(``Isaac-Reorient-Cube-Shadow-Camera-Direct``).
 11
 12Example 1 — Kit viewport (simplest)
 13    One clip from the Kit interactive viewport, showing 4 parallel environments.
 14
 15    .. code-block:: bash
 16
 17        uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
 18            --example 1 --num_envs 4
 19
 20Example 2 — scene sensor only, headless
 21    One clip captured directly from the scene's tiled-camera sensor.
 22    No visualizer window opens; the sensor renders offline.
 23
 24    .. code-block:: bash
 25
 26        uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
 27            --example 2 --num_envs 16
 28
 29Example 3 — Kit viewport + Kit tiled grid + Newton viewport + scene sensor
 30    Four independent clip streams recorded simultaneously.
 31
 32    .. code-block:: bash
 33
 34        uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
 35            --example 3 --num_envs 4
 36
 37Clips are written to ``videos/recording_tutorial/example_<N>/`` in the working directory.
 38Examples 1 and 2 each demonstrate one recording source; Example 3 combines all of them.
 39"""
 40
 41from __future__ import annotations
 42
 43import argparse
 44import contextlib
 45import os
 46import sys
 47
 48import gymnasium as gym
 49import torch
 50
 51import isaaclab_tasks  # noqa: F401
 52
 53with contextlib.suppress(ImportError):
 54    import isaaclab_tasks_experimental  # noqa: F401
 55
 56from isaaclab.app import add_launcher_args, launch_simulation
 57from isaaclab.envs.utils.video_recorder_cfg import VideoRecorderCfg
 58
 59from isaaclab_tasks.utils import resolve_task_config, setup_preset_cli
 60
 61# ---------------------------------------------------------------------------
 62# Constants
 63# ---------------------------------------------------------------------------
 64
 65_VIDEO_LENGTH = 100  # env steps per clip
 66_NUM_STEPS = 115  # slightly more than _VIDEO_LENGTH so the clip flushes cleanly
 67
 68_TASK_SHADOW = "Isaac-Reorient-Cube-Shadow-Camera-Direct"
 69
 70# Kit viewport camera: positioned to show a 2×2 grid of Shadow Hand environments.
 71# env_spacing=1.0 with 4 envs → envs centered at ±0.5 in x and y.
 72# Cube spawns at ~(0, -0.39, 0.6) per env; wrist cylinder is the landmark at the top.
 73_SHADOW_EYE = (0.0, -2.2, 1.8)
 74_SHADOW_LOOKAT = (0.0, -0.1, 0.4)
 75_SHADOW_ENV_SPACING = 1.0
 76
 77# Skip the first few steps so the RTX renderer has warmed up before recording starts.
 78_KIT_STEP_OFFSET = 5
 79
 80
 81def _output_dir(example: int) -> str:
 82    return os.path.join("videos", "recording_tutorial", f"example_{example}")
 83
 84
 85def _shadow_env_cfg(num_envs: int, env_spacing: float = _SHADOW_ENV_SPACING):
 86    """Build a base Shadow Hand camera env cfg shared by all examples."""
 87    env_cfg, _ = resolve_task_config(_TASK_SHADOW, "", overrides=(*sys.argv[1:], "env.scene.tiled_camera=rgb"))
 88    env_cfg.scene.tiled_camera.height = 256
 89    env_cfg.scene.tiled_camera.width = 256
 90    env_cfg.scene.num_envs = num_envs
 91    env_cfg.scene.env_spacing = env_spacing
 92    return env_cfg
 93
 94
 95# ---------------------------------------------------------------------------
 96# Per-example environment config builders
 97# ---------------------------------------------------------------------------
 98
 99
100def _build_env_cfg_example_1(num_envs: int):
101    """Shadow Hand + Kit viewport: one clip from the interactive viewport."""
102    from isaaclab_visualizers.kit import KitVisualizerCfg
103
104    env_cfg = _shadow_env_cfg(num_envs)
105    env_cfg.sim.visualizer_cfgs = [KitVisualizerCfg(eye=_SHADOW_EYE, lookat=_SHADOW_LOOKAT)]
106
107    out = _output_dir(1)
108    env_cfg.video_recorders = [
109        VideoRecorderCfg(
110            source="visualizer:kit",
111            output_dir=out,
112            output_filename_prefix="kit_viewport",
113            video_length=_VIDEO_LENGTH,
114            fps=30,
115            step_offset=_KIT_STEP_OFFSET,
116        ),
117    ]
118    return env_cfg, _TASK_SHADOW
119
120
121def _build_env_cfg_example_2(num_envs: int):
122    """Shadow Hand + headless: scene tiled-camera sensor clip only."""
123    env_cfg = _shadow_env_cfg(num_envs, env_spacing=2.0)
124    env_cfg.sim.visualizer_cfgs = []  # no interactive visualizer
125
126    out = _output_dir(2)
127    env_cfg.video_recorders = [
128        VideoRecorderCfg(
129            source="sensor:tiled_camera",
130            output_dir=out,
131            output_filename_prefix="sensor",
132            video_length=_VIDEO_LENGTH,
133            fps=30,
134        ),
135    ]
136    return env_cfg, _TASK_SHADOW
137
138
139def _build_env_cfg_example_3(num_envs: int):
140    """Shadow Hand + Kit viewport + Kit tiled grid + Newton viewport + sensor: four simultaneous streams.
141
142    Note: ``source='visualizer:newton'`` captures the full Newton GL window. When
143    ``streaming_view=True`` is set on :class:`~isaaclab_visualizers.newton.NewtonGLVisualizerCfg`,
144    the GL window displays the per-environment camera panel, so this effectively records
145    a Newton streaming view without a separate ``render_tiled_rgb_array()`` call.
146    """
147    from isaaclab_visualizers.kit import KitVisualizerCfg
148    from isaaclab_visualizers.newton import NewtonGLVisualizerCfg
149
150    env_cfg = _shadow_env_cfg(num_envs)
151    kit_cfg = KitVisualizerCfg(
152        eye=_SHADOW_EYE,
153        lookat=_SHADOW_LOOKAT,
154        streaming_view=True,
155        streaming_envs=min(num_envs, 16),
156        # No streaming_sensor_prim_path/streaming_cam_target_prim_path: adopts the existing
157        # tiled_camera sensor automatically, so the streaming panel shows the same
158        # RTX-rendered views as source="sensor:tiled_camera".
159    )
160    newton_cfg = NewtonGLVisualizerCfg(
161        eye=_SHADOW_EYE,
162        lookat=_SHADOW_LOOKAT,
163        window_width=1280,
164        window_height=720,
165        focal_length=25.0,
166    )
167    env_cfg.sim.visualizer_cfgs = [kit_cfg, newton_cfg]
168
169    out = _output_dir(3)
170    env_cfg.video_recorders = [
171        VideoRecorderCfg(
172            source="visualizer:kit",
173            output_dir=out,
174            output_filename_prefix="kit_viewport",
175            video_length=_VIDEO_LENGTH,
176            fps=30,
177            step_offset=_KIT_STEP_OFFSET,
178        ),
179        VideoRecorderCfg(
180            source="visualizer:kit:streaming_view",
181            output_dir=out,
182            output_filename_prefix="tiled_kit_viewport",
183            video_length=_VIDEO_LENGTH,
184            fps=30,
185            step_offset=_KIT_STEP_OFFSET,
186        ),
187        VideoRecorderCfg(
188            source="visualizer:newton",
189            output_dir=out,
190            output_filename_prefix="newton_viewport",
191            video_length=_VIDEO_LENGTH,
192            fps=30,
193        ),
194        VideoRecorderCfg(
195            source="sensor:tiled_camera",
196            output_dir=out,
197            output_filename_prefix="sensor",
198            video_length=_VIDEO_LENGTH,
199            fps=30,
200        ),
201    ]
202    return env_cfg, _TASK_SHADOW
203
204
205_BUILDERS = {
206    1: _build_env_cfg_example_1,
207    2: _build_env_cfg_example_2,
208    3: _build_env_cfg_example_3,
209}
210
211# ---------------------------------------------------------------------------
212# Argument parsing
213# ---------------------------------------------------------------------------
214parser = argparse.ArgumentParser(description="Video recording tutorial for Isaac Lab environments.")
215parser.add_argument(
216    "--example", type=int, default=1, choices=[1, 2, 3], help="Which recording example to run (1, 2, or 3)."
217)
218parser.add_argument("--num_envs", type=int, default=None, help="Number of environments to simulate.")
219add_launcher_args(parser)
220args_cli, hydra_args = setup_preset_cli(parser)
221sys.argv = [sys.argv[0]] + hydra_args
222
223
224def main():
225    """Run the selected video recording example."""
226    defaults = {1: 4, 2: 16, 3: 4}
227    num_envs = args_cli.num_envs if args_cli.num_envs is not None else defaults[args_cli.example]
228    env_cfg, task = _BUILDERS[args_cli.example](num_envs)
229    env_cfg.sim.device = args_cli.device if args_cli.device is not None else env_cfg.sim.device
230
231    # Examples 1 and 3 record from the Kit viewport via omni.replicator, which requires
232    # camera rendering support.  Force it here for visualizer-only recording.
233    if args_cli.example in (1, 3):
234        args_cli.enable_cameras = True
235
236    with launch_simulation(env_cfg, args_cli):
237        env = gym.make(task, cfg=env_cfg)
238
239        out = _output_dir(args_cli.example)
240        print(f"[INFO]: Running Example {args_cli.example} — clips → {out}/")
241        print(f"[INFO]: Gym observation space: {env.observation_space}")
242        print(f"[INFO]: Gym action space: {env.action_space}")
243
244        print("[INFO]: Setup complete.")
245        env.reset()
246        for _ in range(_NUM_STEPS):
247            with torch.inference_mode():
248                actions = 2 * torch.rand(env.action_space.shape, device=env.unwrapped.device) - 1
249                env.step(actions)
250
251        env.close()
252        print(f"[INFO]: Done. Clips written to {out}/")
253
254
255if __name__ == "__main__":
256    main()

Example 1: Kit viewport#

uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
    --example 1 --num_envs 4 --viz kit

  • Records the Kit interactive viewport (RTX renderer)

  • Shows 4 parallel environments

  • One clip is written to videos/recording_tutorial/example_1/kit_viewport_0000.mp4

Kit visualizer

Example 2: Scene sensor, headless#

uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
    --example 2 --num_envs 16

Scene sensor

  • No visualizer window opens; frames are read directly from the tiled_camera sensor

  • One clip is written to videos/recording_tutorial/example_2/sensor_0000.mp4

  • source="sensor:tiled_camera" is the key under which the camera is registered in env.scene.sensors

  • The sensor must have the recorded channel in its data_types; rgb is the default, and depth, segmentation, and normals are also available (see Sources)

Example 3: All sources simultaneously#

uv run python scripts/tutorials/07_visualizers/run_video_recording.py \
    --example 3 --num_envs 4 --viz kit,newton_gl

Four independent clips are written to videos/recording_tutorial/example_3/:

  • kit_viewport_0000.mp4: Kit interactive viewport (RTX renderer)

  • tiled_kit_viewport_0000.mp4: Kit tiled-camera grid (per-environment views)

  • newton_viewport_0000.mp4: Newton GL viewer framebuffer

  • sensor_0000.mp4: scene tiled-camera sensor (offline render)

Newton GL visualizer

Kit visualizer tiled streaming

Recipes#

The recipes below are command-line and configuration snippets that are not part of the tutorial script.

Recording during RL training#

Record a short clip at regular intervals while training, without opening a window:

uv run --extra isaacsim --extra video isaaclab train --rl_library rsl_rl \
    --task Isaac-Cartpole --video \
    --video_length 200 --video_interval 2000

With no --viz or configured visualizer, --video records from a headless Kit visualizer. This writes a 200-step clip every 2,000 steps. The train and play entrypoints put clips in videos/ under the run’s log directory unless a recorder sets output_dir; a script that builds its own recorders writes to output_dir (default videos). Add --viz kit to watch the run in a window while recording the same view.

Recording depth or segmentation from a scene sensor#

Append a channel to the sensor source to record something other than RGB. Each recorder writes its own file, so several channels of one camera can be recorded in the same run:

env_cfg.video_recorders = [
    VideoRecorderCfg(source="sensor:tiled_camera:rgb", output_filename_prefix="rgb"),
    VideoRecorderCfg(source="sensor:tiled_camera:depth", output_filename_prefix="depth"),
]

Depth uses the turbo colormap over depth_colormap_min to depth_colormap_max. Use the same prefix-per-recorder pattern for segmentation and normals.

Recording while streaming with Viser or Rerun#

Viser and Rerun stream to a browser or viewer and have no frame capture, so they cannot be recorded. Pair them with a capture-capable visualizer that runs headless next to them:

from isaaclab.envs.utils.video_recorder_cfg import VideoRecorderCfg
from isaaclab_visualizers.newton import NewtonGLVisualizerCfg
from isaaclab_visualizers.viser import ViserVisualizerCfg

env_cfg.sim.visualizer_cfgs = [
    ViserVisualizerCfg(),
    NewtonGLVisualizerCfg(headless=True),
]
env_cfg.video_recorders = [VideoRecorderCfg(source="visualizer:newton_gl")]

Alternatively, record directly from a scene camera with source="sensor:<name>", which needs no visualizer.

Recording from an independent camera angle#

Configure the recording angle on the visualizer, not the recorder. To record from a headless Newton visualizer at a different angle alongside an interactive Kit viewer, configure both and omit --viz so the configuration selects them:

from isaaclab_visualizers.kit import KitVisualizerCfg
from isaaclab_visualizers.newton import NewtonGLVisualizerCfg

env_cfg.sim.visualizer_cfgs = [
    KitVisualizerCfg(eye=(4.0, 4.0, 2.0)),
    NewtonGLVisualizerCfg(eye=(12.0, 0.0, 6.0), headless=True),
]
env_cfg.video_recorders = [
    VideoRecorderCfg(source="visualizer:newton_gl", output_dir="videos/"),
]

Alternatively, use a CameraCfg sensor in the scene and record with source="sensor:<name>", which gives full control over the recording viewpoint without requiring a second interactive visualizer.

Sources#

The source string selects what to capture:

Source string

Captures from

"visualizer" (default)

First active recording-capable visualizer

"visualizer:kit"

Kit visualizer viewport

"visualizer:kit:streaming_view"

Kit streaming camera panel, requires streaming_view=True

"visualizer:newton_gl"

Newton GL visualizer viewport

"visualizer:newton_rtx"

Newton OVRTX path-traced viewport

"visualizer:newton_gl:streaming_view"

Newton GL streaming camera panel, requires streaming_view=True

"sensor:<name>"

env.scene.sensors[name], RGB (default)

"sensor:<name>:rgb"

RGB channel

"sensor:<name>:depth"

Depth, turbo colormap, range depth_colormap_min … depth_colormap_max

"sensor:<name>:segmentation"

Segmentation, colorized

"sensor:<name>:normals"

Surface normals, colorized

The camera angle, resolution, and other visualizer settings are configured on the corresponding visualizer config, not on the recorder. A named visualizer source must also have a matching active visualizer in sim.visualizer_cfgs. Set its headless=True when it should capture without a window. The deprecated visualizer:newton source still selects Newton GL; use visualizer:newton_gl.

Note

The Newton RTX viewer framebuffer can be recorded with "visualizer:newton_rtx", but recording its streaming view is not supported.

Clip control#

Field

Default

Meaning

video_length

200

Env steps per clip

video_interval

0

0 = one clip starting at step 1; N > 0 = new clip every N steps

fps

None

Output frame rate; None resolves from env.metadata["render_fps"] or 1 / step_dt

output_dir

"videos"

Directory for output files (created on demand)

output_filename_prefix

"clip"

File stem; output is <prefix>_NNNN.mp4

keep_last_n_clips

None

Delete older clips; None keeps all

One clip at the start of a run:

VideoRecorderCfg(source="visualizer:kit", video_length=500, video_interval=0)

Recurring clips every 1,000 env steps:

VideoRecorderCfg(source="visualizer:kit", video_length=200, video_interval=1000)

Keep only the most recent clip on disk:

VideoRecorderCfg(source="visualizer:kit", video_length=200, video_interval=1000,
                 keep_last_n_clips=1)

Limitations#

  • source="visualizer:kit" and source="visualizer:kit:streaming_view" require cubric to propagate Newton Fabric scene transforms to the RTX renderer. Without cubric, a warning is logged and a black-frame warning is emitted at clip write time. Use source="visualizer:newton_gl" for guaranteed capture with Newton physics.

  • source="visualizer:newton_gl:streaming_view" and source="visualizer:kit:streaming_view" require streaming_view=True on the corresponding visualizer cfg. A RuntimeError is raised at the first capture attempt if it is not set.

  • For source="sensor:<name>", the named sensor must exist on the scene config with the recorded channel ("rgb" by default) in its data_types.

Visualizer

--video

Notes

kit

✓

Launches Isaac Sim; the launch enables camera rendering for the recording.

newton_gl

✓

Uses pyglet’s EGL backend when headless; select it with --viz newton_gl or configure a NewtonGLVisualizerCfg.

newton_rtx

✓

Starts the OVRTX runtime; capture performs a GPU-to-CPU readback of the path-traced framebuffer.

rerun

✗

Remote streaming tool; no local frame-capture API. --video --viz rerun raises an error; select it together with a capture-capable visualizer to record.

viser

✗

Browser streaming tool; no local frame-capture API. --video --viz viser raises an error; select it together with a capture-capable visualizer to record.

To record video while streaming with Rerun or Viser, see Recording while streaming with Viser or Rerun.

See also#