isaaclab.renderers

Sub-package for renderer configurations and implementations.

This sub-package contains configuration classes and implementations for different renderer backends that can be used with Isaac Lab.

Classes

BaseRenderer	Abstract base class for renderer implementations.
RendererCfg	Configuration for a renderer.

Base Renderer

class isaaclab.renderers.BaseRenderer

Bases: ABC

Abstract base class for renderer implementations.

Methods:

initialize()	Post-physics one-time initialization hook.
prepare_cameras(stage, spec)	Pre-render per-camera setup the backend needs.
supported_output_types()	Per-output layout (channels + dtype) this renderer can produce.
prepare_stage(stage, num_envs)	Prepare the stage for rendering before create_render_data() is called.
create_render_data(spec)	Create render data for the given camera CameraRenderSpec.
set_outputs(render_data, output_data)	Store reference to output buffers for writing during render.
update_transforms()	Update scene transforms before rendering.
update_geometries()	Update mutable geometry attributes before rendering.
update_camera(render_data, positions, ...)	Update camera poses and intrinsics for the next render.
render(render_data)	Perform rendering and write to output buffers.
read_output(render_data, camera_data)	Read rendered outputs from the renderer into the camera data container.
cleanup(render_data)	Release renderer resources associated with the given render data.
close()	Release resources owned by the renderer itself rather than by a render data.

Attributes:

visual_material_writer

Return the backend's shared material-writer factory, if supported.

initialize() → None

Post-physics one-time initialization hook. Called only once.

property visual_material_writer: Callable[[tuple[VisualMaterialBatch, ...]], Any] | None

Return the backend’s shared material-writer factory, if supported.

Its writer accepts None for a full sync or channel-to-material-offset device arrays plus one environment-id device array for partial writes, and provides an idempotent close().

prepare_cameras(stage: Any, spec: CameraRenderSpec) → None

Pre-render per-camera setup the backend needs.

The default implementation is a no-op. Renderer subclasses override to perform whatever per-camera initialization their backend requires — e.g. authoring stage attributes on the resolved camera prims, configuring per-tile GPU buffers, or any other state setup.

Parameters:

• stage – Scene stage the camera prims live on, or None when no stage context applies. Stage-less backends ignore it.

• spec – Immutable description of the tiled camera bundle.

abstractmethod supported_output_types() → dict[RenderBufferKind, RenderBufferSpec]

Per-output layout (channels + dtype) this renderer can produce.

Outputs absent from the mapping are not produced by this backend.

Returns:

Mapping from supported RenderBufferKind to its RenderBufferSpec.

abstractmethod prepare_stage(stage: Any, num_envs: int) → None

Prepare the stage for rendering before create_render_data() is called.

Some renderers need to export or preprocess the USD stage before creating render data. This method is called after the renderer is instantiated and before create_render_data().

Parameters:

• stage – USD stage to prepare, or None if not applicable.

• num_envs – Number of environments.

abstractmethod create_render_data(spec: CameraRenderSpec) → Any

Create render data for the given camera CameraRenderSpec.

Parameters:

spec – Immutable description of the tiled camera (paths, config, device).

Returns:

Renderer-specific data for subsequent render() / read_output() calls.

abstractmethod set_outputs(render_data: Any, output_data: dict[str, ProxyArray]) → None

Store reference to output buffers for writing during render.

Parameters:

• render_data – The render data object from create_render_data().

• output_data – Dictionary mapping output names (e.g. "rgb", "depth") to pre-allocated ProxyArray wrappers where rendered data will be written. Use .warp for the underlying warp array or .torch for a zero-copy tensor view.

abstractmethod update_transforms() → None

Update scene transforms before rendering.

Called to sync physics/asset pose state into the renderer’s scene representation.

abstractmethod update_geometries() → None

Update mutable geometry attributes before rendering.

Called to sync physics-driven geometry such as mesh points, extents, or other per-frame geometry buffers into the renderer’s scene representation.

abstractmethod update_camera(render_data: Any, positions: ProxyArray, orientations: ProxyArray, intrinsics: ProxyArray) → None

Update camera poses and intrinsics for the next render.

Parameters:

• render_data – The render data object from create_render_data().

• positions – Camera positions in world frame. Shape (N,), dtype wp.vec3f. Use .torch for a (N, 3) tensor view.

• orientations – Camera orientations as quaternions (x, y, z, w). Shape (N,), dtype wp.quatf. Use .torch for a (N, 4) tensor view.

• intrinsics – Camera intrinsic matrices. Shape (N,), dtype wp.mat33f. Use .torch for a (N, 3, 3) tensor view.

abstractmethod render(render_data: Any) → None

Perform rendering and write to output buffers.

Parameters:

render_data – The render data object from create_render_data().

abstractmethod read_output(render_data: Any, camera_data: CameraData) → None

Read rendered outputs from the renderer into the camera data container.

Parameters:

• render_data – The render data object from create_render_data().

• camera_data – The CameraData instance to populate.

abstractmethod cleanup(render_data: Any) → None

Release renderer resources associated with the given render data.

Parameters:

render_data – The render data object to clean up, or None.

close() → None

Release resources owned by the renderer itself rather than by a render data.

A renderer is shared by every camera whose configuration resolves to it (see get_renderer()), so state it owns outlives any single camera and cannot be released from cleanup(). close() calls this once at simulation teardown, while the stage and the underlying renderer backend are still alive.

The default implementation is a no-op, for backends whose state lives entirely on the render data. Implementations must be idempotent.

Renderer Configuration

class isaaclab.renderers.RendererCfg

Bases: object

Configuration for a renderer.

Attributes:

class_type

Renderer implementation class.

class_type: type[BaseRenderer] | str | None

Renderer implementation class. Concrete configs must set this field.

Additional Public Classes

The following classes are part of the public isaaclab.renderers API.

CameraRenderSpec	Stable inputs for create_render_data().
RenderBufferKind	Canonical names for the per-pixel render buffer kinds a renderer can publish.
RenderBufferSpec	Per-pixel layout (channels + dtype) for one render buffer kind.
RenderContext	Own camera renderers and flat runtime material buffers for one simulation.

class isaaclab.renderers.CameraRenderSpec

Bases: object

Stable inputs for create_render_data().

Backends use this instead of holding a reference to the Camera sensor instance, avoiding circular dependencies between sensors and render data.

Parameters:

• cfg – Camera configuration (data types, resolution, filters, etc.).

• device – Torch device string (e.g. "cuda:0") used by GPU annotators and Warp.

• num_instances – Number of tiled camera instances (environments).

• camera_prim_paths – Absolute USD paths for each environment’s camera prim.

• view_count – Number of camera prims (must match len(camera_prim_paths)).

• camera_path_relative_to_env_0 – Camera prim path with /World/envs/env_0/ prefix stripped; required by OVRTX. Empty string if the first camera is not under env 0.

Methods:

__new__(*args, **kwargs)
__init__(cfg, device, num_instances, ...)

classmethod __new__(*args, **kwargs)

__init__(cfg: CameraCfg, device: str, num_instances: int, camera_prim_paths: tuple[str, ...], view_count: int, camera_path_relative_to_env_0: str) → None

class isaaclab.renderers.RenderBufferKind

Bases: StrEnum

Canonical names for the per-pixel render buffer kinds a renderer can publish.

String values match the vocabulary used by isaaclab.sensors.camera.CameraCfg.data_types.

Methods:

__new__(value)
__init__(*args, **kwds)

__new__(value)

__init__(*args, **kwds)

class isaaclab.renderers.RenderBufferSpec

Bases: object

Per-pixel layout (channels + dtype) for one render buffer kind.

Methods:

__new__(*args, **kwargs)
__init__(channels, dtype)

classmethod __new__(*args, **kwargs)

__init__(channels: int, dtype: type) → None

class isaaclab.renderers.RenderContext

Bases: object

Own camera renderers and flat runtime material buffers for one simulation.

A camera reuses a backend when a prior camera registered a config equal under == (value equality) and the same concrete RendererCfg subclass. A distinct RendererCfg that maps to a different implementation (e.g. Isaac RTX vs Newton) produces another backend; each has BaseRenderer.prepare_stage() run before use.

update_scene_state() is invoked at most once per get_physics_step_count() for the context;

Methods:

__init__()
__new__(*args, **kwargs)

__init__() → None

classmethod __new__(*args, **kwargs)