isaaclab.scene_data

Sub-package containing the scene data provider and backend interface.

The SceneDataProvider bridges physics simulation backends and the consumers that read scene transforms (renderers and visualizers). Physics backends implement SceneDataBackend to expose their current transforms in one of the SceneDataFormat Warp struct variants; the provider converts and remaps them on demand for each consumer.

This package is deliberately separate from isaaclab.scene so that physics backends (isaaclab_physx, isaaclab_newton) can subclass SceneDataBackend without pulling isaaclab.scene into the Kit launcher pre-launch import chain.

Classes

SceneDataProvider	Borrow or convert published arrays; producers own native refresh, renderers own destination lifecycle.
SceneDataBackend
SceneDataFormat	Warp struct variants describing the transform layouts that a SceneDataBackend may publish to consumers.

Scene Data Provider

class isaaclab.scene_data.SceneDataProvider

Bases: object

Borrow or convert published arrays; producers own native refresh, renderers own destination lifecycle.

Methods:

__init__(backend)	Initialize the scene data provider.
get_transforms(output[, mapping, ...])	Bind shared transforms or write them directly into caller-owned output arrays.
set_interactive_scene(scene)	Attach the active interactive scene for scene-owned sensor discovery.
get_interactive_scene()	Return the registered interactive scene, if available.
get_camera_sensors()	Return Isaac Lab camera sensors keyed by scene sensor name.
get_contact_sensors()	Return Isaac Lab contact sensors keyed by scene sensor name.
get_camera_transforms()	Per-camera, per-environment world transforms discovered from USD.
init_output(output)	Allocate any uninitialized fields in output with empty Warp arrays.
create_mapping(paths)	Create an index mapping from sim backend transforms to desired output ordering.
get_geometry_points(*[, output, offsets])	Borrow visual point views or convert directly into the requested native destination.

Attributes:

transform_count	Number of transforms available from the sim backend.
num_envs	Number of environments discovered from /World/envs/env_<id> prims.

__init__(backend: SceneDataBackend)

Initialize the scene data provider.

Parameters:

backend – The simulation backend that supplies raw transform data.

get_transforms(output: SceneDataFormat.Vec3_Quat | SceneDataFormat.Transform | SceneDataFormat.Matrix44 | SceneDataFormat.Vec3_Matrix33 | SceneDataFormat.TransposedMatrix44d | SceneDataFormat.FabricMatrix44, mapping: wp.array | wp.fabricarray | None = None, allow_passthrough: bool = True, *, count: int | None = None, scales: wp.array | None = None) → bool

Bind shared transforms or write them directly into caller-owned output arrays.

With passthrough enabled, matching native arrays are borrowed without a copy; other layouts share SDP-owned buffers converted once per producer timestamp. Treat these arrays as read-only. With passthrough disabled, conversion writes directly into output. Fabric destinations must already be bound by their rendering owner.

Parameters:

• output – A SceneDataFormat struct instance specifying the requested format. Missing non-Fabric arrays are allocated when passthrough is disabled.

• mapping – Native-to-output indices from create_mapping(), or identity ordering. Fabric destinations use their native output-to-source index attribute.

• allow_passthrough – Whether to bind shared arrays instead of writing caller-owned arrays.

• count – Destination count when remapping, or the native transform count.

• scales – Static output scales for TransposedMatrix44d, shape [count], or source-indexed authored scales for Fabric. Mapping and scales are immutable for a binding’s lifetime; replace their arrays when the layout changes.

Returns:

True if transforms are available in output, False if no transforms are published or the format conversion is unsupported.

set_interactive_scene(scene: Any) → None

Attach the active interactive scene for scene-owned sensor discovery.

get_interactive_scene() → Any | None

Return the registered interactive scene, if available.

get_camera_sensors() → dict[str, Any]

Return Isaac Lab camera sensors keyed by scene sensor name.

get_contact_sensors() → dict[str, Any]

Return Isaac Lab contact sensors keyed by scene sensor name.

property transform_count: int

Number of transforms available from the sim backend.

property num_envs: int

Number of environments discovered from /World/envs/env_<id> prims.

Cached on first call. Returns 0 when no USD stage is available or when no /World/envs/env_<id> prims exist.

get_camera_transforms() → dict[str, Any] | None

Per-camera, per-environment world transforms discovered from USD.

Returns:

Dictionary with keys order (list of template prim paths using env_%d), positions and orientations (per-camera, per-env lists, with None for absent envs), and num_envs. Returns None when no USD stage is available.

init_output(output: Vec3_Quat | Transform | Matrix44 | Vec3_Matrix33)

Allocate any uninitialized fields in output with empty Warp arrays.

Only fields that are currently None are allocated; already-initialized fields are left untouched.

Parameters:

output – A SceneDataFormat struct whose None-valued fields will be replaced with empty arrays of length transform_count.

create_mapping(paths: list[str | None]) → wp.array(dtype=wp.int32) | None

Create an index mapping from sim backend transforms to desired output ordering.

For each transform in the sim backend, the resulting array stores the index into paths where that transform should be written. Transforms whose path does not appear in paths (or maps to None) receive an index of -1 and are skipped during conversion.

Named paths must be unique in both layouts. Binding validates this once per layout pair; independent consumers reuse the mapping and share SDP conversions.

Parameters:

paths – Desired output ordering expressed as prim paths. Use None for slots that should not receive any transform.

Returns:

A Warp int32 array of length transform_count containing the remapped indices, or None for identical layouts.

Raises:

• ValueError – A source or named destination path is repeated.

• KeyError – A named destination has no native publication.

get_geometry_points(*, output: wp.array | SceneDataFormat.FabricPoints | None = None, offsets: dict[str, int] | None = None) → dict[str, wp.array] | wp.array | SceneDataFormat.FabricPoints

Borrow visual point views or convert directly into the requested native destination.

Producers supply exact visual prim paths, native pointers and immutable interpolation metadata. SDP performs interpolation and destination reordering together, once per update timestamp and output layout. Only cross-device destinations require staging.

Parameters:

• output – Consumer-owned world-space point buffer [m] or native Fabric destination. Omit to borrow shared point views. FabricPoints without offsets borrows native Fabric storage when the producer publishes it.

• offsets – Visual prim paths mapped to flat-buffer offsets or Fabric array indices. Keep this mapping immutable for the destination’s lifetime.

Returns:

Read-only world-space views by visual prim path when no output is supplied; otherwise the supplied destination, populated directly on the same device.

Scene Data Backend

class isaaclab.scene_data.SceneDataBackend

Bases: object

Attributes:

geometry_timestamp	Logical update timestamp, advanced after native writes or buffer swaps, including within one step.
native_geometry_formats	Geometry formats the producer can publish without conversion.
transforms_timestamp	Logical timestamp, incremented after native writes or buffer swaps; never reset by readers.
native_transform_formats	Formats available without conversion, used when binding consumer destinations.
transforms	Return native transforms without copying; pointer changes must increment transforms_timestamp.
transform_count	Return the number of transforms in the sim backend.
transform_paths	Return unique native body paths in publication order, fixed for each initialized layout.

Methods:

get_geometry_batches([output_format])	Publish native arrays and exact visual-prim ranges established during construction.
get_transforms(output_format)	Publish the requested native format when available, otherwise the primary format.

geometry_timestamp: int = 0

Logical update timestamp, advanced after native writes or buffer swaps, including within one step.

This is not elapsed simulation time. Cached outputs record the timestamp they contain; reading one output never clears another output’s pending update.

property native_geometry_formats: tuple[Any, ...]

Geometry formats the producer can publish without conversion.

get_geometry_batches(output_format: ~typing.Any = <class 'isaaclab.scene_data.scene_data_backend.SceneDataFormat.Points'>) → list[tuple[Any, dict[str, tuple[int, int]]]]

Publish native arrays and exact visual-prim ranges established during construction.

Each batch pairs a native format with path: (offset, count) ranges. Offsets index native points for Points or output vertices for an interpolated format. Publish the requested native format when available, otherwise the primary formats for SDP to convert. Native FabricPoints uses an empty range dictionary because Fabric owns the indexing.

transforms_timestamp: int

Logical timestamp, incremented after native writes or buffer swaps; never reset by readers.

property native_transform_formats: tuple[Any, ...]

Formats available without conversion, used when binding consumer destinations.

get_transforms(output_format: Any) → Any

Publish the requested native format when available, otherwise the primary format.

property transforms: Vec3_Quat | Transform | Matrix44 | Vec3_Matrix33

Return native transforms without copying; pointer changes must increment transforms_timestamp.

property transform_count: int

Return the number of transforms in the sim backend.

property transform_paths: list[str]

Return unique native body paths in publication order, fixed for each initialized layout.

class isaaclab.scene_data.SceneDataFormat

Bases: object

Warp struct variants describing the transform layouts that a SceneDataBackend may publish to consumers.

Classes:

Vec3_Quat	Separate position and quaternion arrays.
Vec3_Matrix33	Separate position and rotation-matrix arrays.
Transform	Packed warp transforms (position + quaternion).
Matrix44	Packed 4x4 homogeneous transform matrices.
TransposedMatrix44d	Double-precision row-vector transforms, as consumed by USD renderers.
FabricMatrix44	Double-precision row-vector matrices in native Fabric storage.
Points	Flat world-space nodal or particle positions.
WeightedPoints	Visual vertices interpolated from four native simulation nodes.
CapsuleEndpoints	Polyline vertices derived from native capsule poses.
FabricPoints	Native Fabric point arrays, already owned and updated by physics.

class Vec3_Quat

Bases: object

Separate position and quaternion arrays.

Attributes:

positions	Per-transform positions [m].
orientations	Per-transform orientations as quaternions.

positions: warp.array = None

Per-transform positions [m].

orientations: warp.array = None

Per-transform orientations as quaternions.

class Vec3_Matrix33

Bases: object

Separate position and rotation-matrix arrays.

Attributes:

positions	Per-transform positions [m].
orientations	Per-transform orientations as 3x3 rotation matrices.

positions: warp.array = None

Per-transform positions [m].

orientations: warp.array = None

Per-transform orientations as 3x3 rotation matrices.

class Transform

Bases: object

Packed warp transforms (position + quaternion).

Attributes:

transforms

Per-transform packed position + orientation transforms [m, -].

transforms: warp.array = None

Per-transform packed position + orientation transforms [m, -].

class Matrix44

Bases: object

Packed 4x4 homogeneous transform matrices.

Attributes:

matrices

Per-transform 4x4 homogeneous transform matrices [m].

matrices: warp.array = None

Per-transform 4x4 homogeneous transform matrices [m].

class TransposedMatrix44d

Bases: object

Double-precision row-vector transforms, as consumed by USD renderers.

Attributes:

matrices

World transforms [m], shape [transform_count].

matrices: warp.array = None

World transforms [m], shape [transform_count].

class FabricMatrix44

Bases: object

Double-precision row-vector matrices in native Fabric storage.

Attributes:

matrices

Transforms [m], shape [transform_count].

matrices: warp.fabricarray = None

Transforms [m], shape [transform_count].

class Points

Bases: object

Flat world-space nodal or particle positions.

Attributes:

points

World-space positions [m], shape [point_count].

points: warp.array = None

World-space positions [m], shape [point_count].

class WeightedPoints

Bases: object

Visual vertices interpolated from four native simulation nodes.

Attributes:

points	Native world-space simulation nodes [m].
indices	Four native node indices per visual vertex.
weights	Four barycentric weights per visual vertex.

points: warp.array

Native world-space simulation nodes [m].

indices: warp.array2d

Four native node indices per visual vertex.

weights: warp.array2d

Four barycentric weights per visual vertex.

class CapsuleEndpoints

Bases: object

Polyline vertices derived from native capsule poses.

Attributes:

transforms	World-space body poses [m, quaternion].
shape_body
shape_transform	Capsule poses relative to their bodies [m, quaternion].
shape_scale	Capsule scales [m]; the second component is the half-length.
endpoints	Two (capsule index, endpoint sign) pairs per output vertex; their positions are averaged.

transforms: warp.array

World-space body poses [m, quaternion].

shape_body: warp.array

shape_transform: warp.array

Capsule poses relative to their bodies [m, quaternion].

shape_scale: warp.array

Capsule scales [m]; the second component is the half-length.

endpoints: warp.array

Two (capsule index, endpoint sign) pairs per output vertex; their positions are averaged.

class FabricPoints

Bases: object

Native Fabric point arrays, already owned and updated by physics.

Attributes:

points

Per-prim native point storage [m].

points: warp.fabricarrayarray

Per-prim native point storage [m].