isaaclab_teleop

Package providing IsaacTeleop-based teleoperation for Isaac Lab.

Classes

IsaacTeleopCfg	Configuration for IsaacTeleop-based teleoperation.
IsaacTeleopDevice	A IsaacTeleop-based teleoperation device for Isaac Lab.
HapticFeedbackCfg	Base configuration for teleop haptic feedback.
ControllerHapticFeedbackCfg	Haptic feedback rendered as XR motion-controller vibration.
GloveHapticFeedbackCfg	Haptic feedback rendered as per-finger power on a haptic glove.
HapticFeedbackReceiver	Protocol for a teleop device that can render haptic feedback.
HapticFeedbackDriver	Reads the per-hand signal from the scene and pushes it to the device.
XrCfg	Configuration for viewing and interacting with the environment through an XR device.
XrAnchorRotationMode	Enumeration for XR anchor rotation modes.
XrAnchorSynchronizer	Keeps the XR anchor prim aligned with a reference prim according to XR config.

Functions

create_isaac_teleop_device(cfg[, ...])	Create an IsaacTeleopDevice with required Omniverse extension setup.
create_haptic_feedback_driver(env, device, ...)	Build a HapticFeedbackDriver when the env and device support haptics.
remove_camera_configs(env_cfg)	Removes cameras from environments when using XR devices.

Configuration

class isaaclab_teleop.IsaacTeleopCfg

Configuration for IsaacTeleop-based teleoperation.

This configuration class defines the parameters needed to create a IsaacTeleop teleoperation session integrated with Isaac Lab environments.

The pipeline_builder is a callable that constructs the IsaacTeleop retargeting pipeline. It should return an OutputCombiner with a single “action” output that contains the flattened action tensor (typically via TensorReorderer).

If the pipeline builder also produces retargeters that should be exposed in the tuning UI, the env cfg should call the builder, unpack the results, and populate both pipeline_builder and retargeters_to_tune explicitly. Both fields must be callables (lambdas / functions) so they survive the deepcopy performed by @configclass on mutable attributes.

Example

def build_pipeline():
    controllers = ControllersSource(name="controllers")
    se3 = Se3AbsRetargeter(cfg, name="ee_pose")
    # ... connect and flatten with TensorReorderer ...
    pipeline = OutputCombiner({"action": reorderer.output("output")})
    return pipeline, [se3]  # return retargeters separately


pipeline, retargeters = build_pipeline()
teleop_cfg = IsaacTeleopCfg(
    xr_cfg=XrCfg(anchor_pos=(0.5, 0.0, 0.5)),
    pipeline_builder=lambda: pipeline,
    retargeters_to_tune=lambda: retargeters,
)

Attributes:

xr_cfg	XR anchor configuration for positioning the user in the simulation.
pipeline_builder	Callable that builds the IsaacTeleop retargeting pipeline.
plugins	List of IsaacTeleop plugin configurations.
sim_device	Torch device string for placing output action tensors.
retargeting_execution	IsaacTeleop retargeting execution settings.
teleoperation_active_default	Whether teleoperation should be active by default when the session starts.
retargeters_to_tune	Optional callable returning retargeters to expose in the tuning UI.
control_channel_uuid	16-byte UUID for the teleop control message channel.
target_frame_prim_path	Optional USD prim path whose world frame becomes the target coordinate frame for all output poses.
app_name	Application name for the IsaacTeleop session.

Methods:

__init__([xr_cfg, pipeline_builder, ...])

xr_cfg: XrCfg

XR anchor configuration for positioning the user in the simulation.

This includes anchor position, rotation, and optional dynamic anchoring to follow a prim (e.g., robot base) during locomotion tasks.

pipeline_builder: Callable[[], OutputCombiner]

Callable that builds the IsaacTeleop retargeting pipeline.

The function should return an OutputCombiner with an “action” output containing the flattened action tensor matching the Isaac Lab action space. Use TensorReorderer to flatten multiple retargeter outputs into a single array.

To expose retargeters for the tuning UI, populate retargeters_to_tune directly when constructing this config rather than encoding them into the builder’s return value.

plugins: list[PluginConfig]

List of IsaacTeleop plugin configurations.

Plugins can provide additional functionality like synthetic hand tracking from controller inputs.

__init__(xr_cfg: XrCfg = <factory>, pipeline_builder: Callable[[], OutputCombiner] = <factory>, plugins: list[PluginConfig] = <factory>, sim_device: str = <factory>, retargeting_execution: RetargetingExecutionConfig | None = <factory>, teleoperation_active_default: bool = <factory>, retargeters_to_tune: Callable[[], list[BaseRetargeter]] | None = <factory>, control_channel_uuid: bytes | None = <factory>, target_frame_prim_path: str | None = <factory>, app_name: str = <factory>) → None

sim_device: str

Torch device string for placing output action tensors.

retargeting_execution: RetargetingExecutionConfig | None

IsaacTeleop retargeting execution settings.

Left as None by default so that importing and constructing this config never requires the optional isaacteleop package (e.g. on platforms where it is not installed). When None, Isaac Lab resolves it at session start to IsaacTeleop’s pipelined, deadline-paced default (RetargetingExecutionConfig(mode="pipelined", pacing=DeadlinePacingConfig(safety_margin_s=0.025))), where isaacteleop is guaranteed to be available. Set this explicitly to RetargetingExecutionConfig(mode="sync") for exact current-frame retargeting while debugging or comparing behavior.

teleoperation_active_default: bool

Whether teleoperation should be active by default when the session starts.

When False (the default), the teleop session remains inactive until a "START" command is received from xr_core via the message bus.

retargeters_to_tune: Callable[[], list[BaseRetargeter]] | None

Optional callable returning retargeters to expose in the tuning UI.

Must be a callable (e.g. lambda: [retargeter1, retargeter2]) rather than a plain list because @configclass deep-copies mutable attributes and retargeter objects often contain non-picklable C++/SWIG handles. Wrapping in a callable makes the value opaque to deepcopy.

When set and the tuning UI is enabled, the returned retargeters will be displayed in the MultiRetargeterTuningUIImGui window, allowing real-time adjustment of their tunable parameters. Only retargeters that have a ParameterState (i.e. tunable parameters) will appear.

If None, the tuning UI will not be opened.

control_channel_uuid: bytes | None

16-byte UUID for the teleop control message channel.

Defaults to TELEOP_CONTROL_CHANNEL_UUID (uuid5(NAMESPACE_DNS, "teleop_command")), which is the well-known channel both the Isaac Lab server and CloudXR JS client use to exchange start/stop/reset commands.

When set, a teleop_control_pipeline is created automatically using TeleopMessageProcessor and DefaultTeleopStateManager. The remote client sends UTF-8 control commands over the OpenXR opaque data channel identified by this UUID, and the results are exposed via poll_control_events().

Set to None to disable the control channel entirely.

target_frame_prim_path: str | None

Optional USD prim path whose world frame becomes the target coordinate frame for all output poses.

When set, the device automatically reads this prim’s world transform each frame and uses its inverse as the target_T_world rebase matrix in advance(). An explicit target_T_world argument to advance() takes precedence over this config.

Typical usage: set to the robot base link prim path so that an IK controller receives end-effector poses in the robot’s base frame.

Example:

IsaacTeleopCfg(
    target_frame_prim_path="/World/envs/env_0/Robot/base_link",
    ...
)

app_name: str

Application name for the IsaacTeleop session.

class isaaclab_teleop.XrCfg

Configuration for viewing and interacting with the environment through an XR device.

Attributes:

anchor_pos	Specifies the position (in m) of the simulation when viewed in an XR device.
anchor_rot	Specifies the rotation (as a quaternion xyzw) of the simulation when viewed in an XR device.
anchor_prim_path	Specifies the prim path to attach the XR anchor to for dynamic positioning.
anchor_rotation_mode	Specifies how the XR anchor rotation should behave when attached to a prim.
anchor_rotation_smoothing_time	Wall-clock time constant (seconds) for rotation smoothing in FOLLOW_PRIM_SMOOTHED mode.
anchor_rotation_custom_func	Specifies the function to calculate the rotation of the XR anchor when anchor_rotation_mode is CUSTOM.
near_plane	Specifies the near plane distance for the XR device.
fixed_anchor_height	Specifies if the anchor height should be fixed.

Methods:

__init__([anchor_pos, anchor_rot, ...])

anchor_pos: tuple[float, float, float]

Specifies the position (in m) of the simulation when viewed in an XR device.

Specifically: this position will appear at the origin of the XR device’s local coordinate frame.

anchor_rot: tuple[float, float, float, float]

Specifies the rotation (as a quaternion xyzw) of the simulation when viewed in an XR device.

Specifically: this rotation will determine how the simulation is rotated with respect to the origin of the XR device’s local coordinate frame.

This quantity is only effective if xr_anchor_pos is set.

anchor_prim_path: str | None

Specifies the prim path to attach the XR anchor to for dynamic positioning.

When set, the XR anchor will be attached to the specified prim (e.g., robot root prim), allowing the XR camera to move with the prim. This is particularly useful for locomotion robot teleoperation where the robot moves and the XR camera should follow it.

If None, the anchor will use the static anchor_pos and anchor_rot values.

__init__(anchor_pos: tuple[float, float, float] = <factory>, anchor_rot: tuple[float, float, float, float] = <factory>, anchor_prim_path: str | None = <factory>, anchor_rotation_mode: ~isaaclab_teleop.xr_cfg.XrAnchorRotationMode = <factory>, anchor_rotation_smoothing_time: float = <factory>, anchor_rotation_custom_func: ~collections.abc.Callable[[~numpy.ndarray, ~numpy.ndarray], ~numpy.ndarray] = <factory>, near_plane: float = <factory>, fixed_anchor_height: bool = <factory>) → None

anchor_rotation_mode: XrAnchorRotationMode

Specifies how the XR anchor rotation should behave when attached to a prim.

The available modes are: - XrAnchorRotationMode.FIXED: Sets rotation once to anchor_rot value - XrAnchorRotationMode.FOLLOW_PRIM: Rotation follows prim’s rotation - XrAnchorRotationMode.FOLLOW_PRIM_SMOOTHED: Rotation smoothly follows prim’s rotation using slerp - XrAnchorRotationMode.CUSTOM: user provided function to calculate the rotation

anchor_rotation_smoothing_time: float

Wall-clock time constant (seconds) for rotation smoothing in FOLLOW_PRIM_SMOOTHED mode.

This time constant is applied using wall-clock delta time between frames (not physics dt). Smaller values (e.g., 0.1) result in faster/snappier response but less smoothing. Larger values (e.g., 0.75–2.0) result in slower/smoother response but more lag. Typical useful range: 0.3 – 1.5 seconds depending on runtime frame-rate and comfort.

anchor_rotation_custom_func: Callable[[ndarray, ndarray], ndarray]

Specifies the function to calculate the rotation of the XR anchor when anchor_rotation_mode is CUSTOM.

Parameters:

• headpose – Previous head pose as numpy array [x, y, z, w, x, y, z] (position + quaternion)

• pose – Anchor prim pose as numpy array [x, y, z, w, x, y, z] (position + quaternion)

Returns:

Quaternion as numpy array [w, x, y, z]

Return type:

np.ndarray

near_plane: float

Specifies the near plane distance for the XR device.

This value determines the closest distance at which objects will be rendered in the XR device.

fixed_anchor_height: bool

Specifies if the anchor height should be fixed.

If True, the anchor height will be fixed to the initial height of the anchor prim.

class isaaclab_teleop.XrAnchorRotationMode

Enumeration for XR anchor rotation modes.

Attributes:

FIXED	sets rotation once and doesn't change it.
FOLLOW_PRIM	rotation follows prim's rotation.
FOLLOW_PRIM_SMOOTHED	rotation smoothly follows prim's rotation using slerp.
CUSTOM	user provided function to calculate the rotation.

FIXED = 'fixed'

sets rotation once and doesn’t change it.

Type:

Fixed rotation mode

FOLLOW_PRIM = 'follow_prim'

rotation follows prim’s rotation.

Type:

Follow prim rotation mode

FOLLOW_PRIM_SMOOTHED = 'follow_prim_smoothed'

rotation smoothly follows prim’s rotation using slerp.

Type:

Follow prim rotation mode with smooth interpolation

CUSTOM = 'custom_rotation'

user provided function to calculate the rotation.

Type:

Custom rotation mode

Device

class isaaclab_teleop.IsaacTeleopDevice

Bases: object

A IsaacTeleop-based teleoperation device for Isaac Lab.

This device provides an interface between IsaacTeleop’s retargeting pipeline and Isaac Lab environments. It composes three focused collaborators:

• XrAnchorManager – XR anchor prim setup, synchronization, and coordinate-frame transform computation.

• TeleopSessionLifecycle – pipeline building, OpenXR handle acquisition, session creation/destruction, and action-tensor extraction.

• CommandHandler – callback registration for START / STOP / RESET commands, bridged from the pipeline-based control events.

Together they manage:

1. XR anchor configuration and synchronization

2. IsaacTeleop session lifecycle

3. Action tensor generation from the retargeting pipeline

The device uses IsaacTeleop’s TensorReorderer to flatten pipeline outputs into a single action tensor matching the environment’s action space.

Frame rebasing:

By default, all output poses are expressed in the simulation world frame. When an application needs poses in a different frame (e.g. robot base link for IK), there are two options:

• Config-driven (recommended): set target_frame_prim_path to the USD prim whose frame the output should be expressed in. The device reads the prim’s world transform each frame and applies the rebase automatically.

• Explicit: pass a target_T_world matrix directly to advance().

In both cases the device composes target_T_world @ world_T_anchor before feeding the matrix into the retargeting pipeline, so all resulting poses are expressed in the target frame.

Teleop commands:

The device supports callbacks for START, STOP, and RESET commands that can be triggered via the message-channel control pipeline or registered directly via add_callback().

Example

cfg = IsaacTeleopCfg(
    pipeline_builder=my_pipeline_builder,
    sim_device="cuda:0",
)

# Poses in world frame (default)
with IsaacTeleopDevice(cfg) as device:
    while running:
        action = device.advance()
        env.step(action.repeat(num_envs, 1))

# Config-driven rebase into robot base frame
cfg.target_frame_prim_path = "/World/Robot/base_link"
with IsaacTeleopDevice(cfg) as device:
    while running:
        action = device.advance()
        env.step(action.repeat(num_envs, 1))

# Explicit rebase into robot base frame
with IsaacTeleopDevice(cfg) as device:
    while running:
        robot_T_world = get_robot_base_transform()
        action = device.advance(target_T_world=robot_T_world)
        env.step(action.repeat(num_envs, 1))

Methods:

__init__(cfg[, cloudxr_env_file, ...])	Initialize the IsaacTeleop device.
reset([pause])	Reset the device state.
request_start()	Start teleoperation without an XR client.
request_stop()	Stop (pause) teleoperation without an XR client.
add_callback(key, func)	Add a callback function for teleop commands.
advance([target_T_world])	Process current device state and return control commands.
send_haptic(endpoint, values)	Render one frame of haptic output on a device endpoint.

Attributes:

last_control_events

Control events from the most recent advance().

__init__(cfg: IsaacTeleopCfg, cloudxr_env_file: str | None = None, auto_launch_cloudxr: bool = True, use_kit_xr_bridge: bool = True, mcap_record_path: str | None = None, mcap_replay_path: str | None = None, enable_debug_visualization: bool = False, haptic_cfg: HapticFeedbackCfg | None = None)

Initialize the IsaacTeleop device.

Parameters:

• cfg – Configuration object for IsaacTeleop settings.

• cloudxr_env_file – Optional path to a CloudXR .env file. When provided and auto_launch_cloudxr is True, the CloudXR runtime is launched automatically during session start. When None, no CloudXR runtime is launched.

• auto_launch_cloudxr – Whether to auto-launch the CloudXR runtime when cloudxr_env_file is set. Ignored when cloudxr_env_file is None.

• use_kit_xr_bridge – Whether to source live OpenXR handles from Kit’s XR bridge (True, the full XR rendering / anchor path) or run standalone (False) with isaacteleop owning its own OpenXR session through the CloudXR runtime – teleop I/O with no Kit XR rendering. Typically wired to the --xr CLI flag.

• mcap_record_path – Optional MCAP file path to record the live teleop session into. Mutually exclusive with mcap_replay_path. Debug-grade only – the produced file has no per-episode segmentation, no world-frame anchor, and no public Python decoder.

• mcap_replay_path – Optional MCAP file path to replay. When set, the device runs in SessionMode.REPLAY with no live XR connection and feeds the recorded tracker stream through the pipeline. Mutually exclusive with mcap_record_path.

• enable_debug_visualization – Whether tracking debug visualization (red sphere markers at each OpenXR hand joint, RGB axis markers at the controller aim poses) is enabled at session start. When False (the default), the pipeline carries no visualization overhead.

• haptic_cfg – Optional haptic-feedback configuration. When provided, the device renders per-hand output vectors pushed via send_haptic() on the configured device (controller, glove, …). None disables haptics.

reset(pause: bool = False) → None

Reset the device state.

Resets the XR anchor synchronizer and schedules a reset ExecutionEvents for the next pipeline step so that all retargeters reinitialize their cross-step state. Also clears any pending haptic force so a pulse in progress at reset time does not persist into the next episode.

Parameters:

pause – When True, also pause a running session so teleop resumes from a paused state – the behavior for an operator reset (e.g. keyboard R). Defaults to False for a host reset (e.g. environment auto-reset after task success), which keeps the session running into the next episode.

request_start() → None

Start teleoperation without an XR client.

Drives the internal teleop state machine toward RUNNING (see TeleopSessionLifecycle.request_start()). Useful for headless or keyboard-driven control when no headset UI is available to send START. No-op when no control channel is configured.

request_stop() → None

Stop (pause) teleoperation without an XR client.

Drives the internal teleop state machine to PAUSED (see TeleopSessionLifecycle.request_stop()). No-op when no control channel is configured.

property last_control_events: ControlEvents

Control events from the most recent advance().

Returns a ControlEvents derived from the teleop control pipeline. When no control channel is configured, returns a default (no-op) ControlEvents.

add_callback(key: str, func: Callable) → None

Add a callback function for teleop commands.

Parameters:

• key – The command type to bind to. Valid values are “START”, “STOP”, “RESET”, and “R”.

• func – The function to call when the command is received. Should take no arguments.

advance(target_T_world: np.ndarray | torch.Tensor | SupportsDLPack | None = None) → torch.Tensor | None

Process current device state and return control commands.

If the IsaacTeleop session has not been started yet (because the OpenXR handles were not available at __enter__ time), this method will attempt to start it on each call. Once the user clicks “Start AR” and the handles become available, the session is created transparently.

Parameters:

target_T_world –

Optional 4x4 transform matrix that rebases all output poses into an arbitrary target coordinate frame. When provided, the matrix sent to the retargeting pipeline becomes target_T_world @ world_T_anchor instead of just world_T_anchor, so all resulting poses are expressed in the target frame rather than the simulation world frame.

Typical use case: pass robot_base_T_world so that an IK controller receives end-effector poses in the robot’s base link frame.

Accepts any object supporting the DLPack buffer protocol (__dlpack__), including numpy.ndarray, torch.Tensor, and wp.array.

When None and target_frame_prim_path is set, the transform is computed automatically by reading the prim’s world matrix from Fabric and inverting it.

Returns:

A flattened action torch.Tensor ready for the Isaac Lab environment, or None if the session has not started yet (e.g. still waiting for the user to start AR).

Raises:

RuntimeError – If called outside of a context manager.

send_haptic(endpoint: str, values) → None

Render one frame of haptic output on a device endpoint.

Implements the HapticFeedbackReceiver protocol. The vector is cached and injected into the haptic sink on the next advance(). This is a no-op unless the device was constructed with a haptic_cfg.

Parameters:

• endpoint – "left" or "right" (see ENDPOINT_LEFT).

• values – The per-hand output vector (one scalar for a rumble motor, one value per finger for a glove); an all-zero vector stops feedback.

isaaclab_teleop.create_isaac_teleop_device(cfg: IsaacTeleopCfg, sim_device: str | None = None, callbacks: dict[str, Callable] | None = None, cloudxr_env_file: str | None = None, auto_launch_cloudxr: bool = True, use_kit_xr_bridge: bool = True, mcap_record_path: str | None = None, mcap_replay_path: str | None = None, enable_debug_visualization: bool = False, haptic_cfg: HapticFeedbackCfg | None = None) → IsaacTeleopDevice

Create an IsaacTeleopDevice with required Omniverse extension setup.

This helper centralises the boilerplate that every script must execute before constructing an IsaacTeleopDevice:

1. Disable default OpenXR input bindings (prevents conflicts).

2. Enable the isaacsim.kit.xr.teleop.bridge extension (live mode only – replay mode skips this since it never touches the XR runtime).

3. Optionally override IsaacTeleopCfg.sim_device so action tensors land on the same device the caller uses for the simulation.

Note

When sim_device is provided, cfg.sim_device is mutated in place before the device is constructed.

Parameters:

• cfg – IsaacTeleop configuration.

• sim_device – If provided, overrides cfg.sim_device so action tensors are placed on the requested torch device (e.g. "cuda:0").

• callbacks – Optional mapping of command keys (e.g. "START", "STOP", "RESET") to callables registered on the device.

• cloudxr_env_file – Optional path to a CloudXR .env file. When provided and auto_launch_cloudxr is True, the CloudXR runtime and WSS proxy are launched automatically during session start. When None, no CloudXR runtime is launched.

• auto_launch_cloudxr – Whether to auto-launch the CloudXR runtime when cloudxr_env_file is set. Set to False to skip the launch (e.g. when running the runtime externally). Ignored when cloudxr_env_file is None.

• use_kit_xr_bridge – Whether to drive the session from Kit’s XR bridge (the full XR rendering / anchor path). When True (default) the isaacsim.kit.xr.teleop.bridge extension is enabled and the session sources its OpenXR handles from Kit. When False the session runs standalone – the bridge is left untouched and isaacteleop creates its own OpenXR session through the CloudXR runtime, so teleop I/O works headless without Kit XR rendering. Typically wired to the --xr CLI flag.

• mcap_record_path – Optional MCAP file path to record the live teleop session into. Debug-grade only. Mutually exclusive with mcap_replay_path.

• mcap_replay_path – Optional MCAP file path to replay. When set, the returned device runs in SessionMode.REPLAY and the XR teleop bridge is left untouched. Mutually exclusive with mcap_record_path.

• enable_debug_visualization – Whether tracking debug visualization is enabled at session start. See IsaacTeleopDevice.enable_debug_visualization.

• haptic_cfg – Optional haptic-feedback configuration. When provided, the returned device implements HapticFeedbackReceiver and renders per-hand output vectors on the configured device (controller, glove, …).

Returns:

A fully configured IsaacTeleopDevice ready for use in a with block.

Haptic Feedback

class isaaclab_teleop.HapticFeedbackCfg

Base configuration for teleop haptic feedback.

Device-agnostic. Attach a concrete subclass (ControllerHapticFeedbackCfg or GloveHapticFeedbackCfg) to an environment config as a sibling of isaac_teleop (e.g. self.haptic_feedback = ... in __post_init__). The teleop scripts discover it via hasattr(env_cfg, "haptic_feedback").

A subclass supplies the two pluggable pieces: make_signal_fn() (how the per-hand vector is read from the environment) and build_sink() (how that vector is rendered by an isaacteleop device).

Attributes:

left_sensor_name	Scene entity name of the left-hand ContactSensor.
right_sensor_name	Scene entity name of the right-hand ContactSensor.
num_taxels	1 for a single rumble motor, one channel per finger for a glove.
gain	Signal-to-output gain applied after the deadband (see the subclass docs for units).
deadband	Signal magnitude below which no output is produced (rejects sensor noise).
saturation	Upper clamp on the normalized output amplitude in [0, 1].

Methods:

make_signal_fn()	Return the signal source (environment -> per-hand vector) for this config.
build_sink(force_inputs, tracker_provider)	Build the isaacteleop HapticSink that renders the cached vectors.
__init__([left_sensor_name, ...])

left_sensor_name: str

Scene entity name of the left-hand ContactSensor.

right_sensor_name: str

Scene entity name of the right-hand ContactSensor.

num_taxels: int

1 for a single rumble motor, one channel per finger for a glove.

Type:

Length of the per-hand output vector

gain: float

Signal-to-output gain applied after the deadband (see the subclass docs for units).

deadband: float

Signal magnitude below which no output is produced (rejects sensor noise).

saturation: float

Upper clamp on the normalized output amplitude in [0, 1].

make_signal_fn() → SignalFn

Return the signal source (environment -> per-hand vector) for this config.

build_sink(force_inputs: dict[str, Any], tracker_provider: Callable[[], Any]) → tuple[Any, Any]

Build the isaacteleop HapticSink that renders the cached vectors.

Parameters:

• force_inputs – Per-endpoint graph output nodes, each carrying a TactileVector(num_taxels) fed from the driver every step.

• tracker_provider – Callable returning a DeviceIO tracker to reuse for devices that need one (e.g. a controller). Cross-process devices (e.g. a glove) may ignore it.

Returns:

A (connected_sink, tracker) tuple. connected_sink is the HapticSink.connect(...) result for TeleopSessionConfig(sinks=[...]) (a subgraph, not the sink node). tracker is the device’s DeviceIO tracker (or None); the session uses it to request the device’s OpenXR extensions (e.g. a glove’s XR_NVX1_push_tensor). It is returned separately because the connected subgraph does not expose the device.

__init__(left_sensor_name: str = <factory>, right_sensor_name: str = <factory>, num_taxels: int = <factory>, gain: float = <factory>, deadband: float = <factory>, saturation: float = <factory>) → None

class isaaclab_teleop.ControllerHapticFeedbackCfg

Bases: HapticFeedbackCfg

Haptic feedback rendered as XR motion-controller vibration.

Renders a single scalar per hand (total gripper contact force) as controller rumble via ControllerHapticDevice + TactileVectorToControllerPulse.

Attributes:

num_taxels	1 for a single rumble motor, one channel per finger for a glove.
gain	Force-to-amplitude gain [1/N] applied after the deadband.
deadband	Contact force [N] below which no vibration is produced.
frequency_hz	Vibration frequency [Hz].
duration_s	Pulse duration [s].
left_sensor_name	Scene entity name of the left-hand ContactSensor.
right_sensor_name	Scene entity name of the right-hand ContactSensor.
saturation	Upper clamp on the normalized output amplitude in [0, 1].

Methods:

make_signal_fn()	Return the signal source (environment -> per-hand vector) for this config.
build_sink(force_inputs, tracker_provider)	Build the isaacteleop HapticSink that renders the cached vectors.
__init__([left_sensor_name, ...])

num_taxels: int

1 for a single rumble motor, one channel per finger for a glove.

Type:

Length of the per-hand output vector

gain: float

Force-to-amplitude gain [1/N] applied after the deadband. Default maps ~20 N to full scale.

deadband: float

Contact force [N] below which no vibration is produced.

frequency_hz: float

Vibration frequency [Hz]. 0 selects the XR runtime default.

duration_s: float

Pulse duration [s]. 0 selects the shortest supported pulse; refreshed each frame.

make_signal_fn() → SignalFn

Return the signal source (environment -> per-hand vector) for this config.

build_sink(force_inputs: dict[str, Any], tracker_provider: Callable[[], Any]) → tuple[Any, Any]

Build the isaacteleop HapticSink that renders the cached vectors.

Parameters:

• force_inputs – Per-endpoint graph output nodes, each carrying a TactileVector(num_taxels) fed from the driver every step.

• tracker_provider – Callable returning a DeviceIO tracker to reuse for devices that need one (e.g. a controller). Cross-process devices (e.g. a glove) may ignore it.

Returns:

A (connected_sink, tracker) tuple. connected_sink is the HapticSink.connect(...) result for TeleopSessionConfig(sinks=[...]) (a subgraph, not the sink node). tracker is the device’s DeviceIO tracker (or None); the session uses it to request the device’s OpenXR extensions (e.g. a glove’s XR_NVX1_push_tensor). It is returned separately because the connected subgraph does not expose the device.

__init__(left_sensor_name: str = <factory>, right_sensor_name: str = <factory>, num_taxels: int = <factory>, gain: float = <factory>, deadband: float = <factory>, saturation: float = <factory>, frequency_hz: float = <factory>, duration_s: float = <factory>) → None

left_sensor_name: str

Scene entity name of the left-hand ContactSensor.

right_sensor_name: str

Scene entity name of the right-hand ContactSensor.

saturation: float

Upper clamp on the normalized output amplitude in [0, 1].

class isaaclab_teleop.GloveHapticFeedbackCfg

Bases: HapticFeedbackCfg

Haptic feedback rendered as per-finger power on a haptic glove.

Renders per-finger grip force against the grasped object as finger vibration via a cross-process haptic_glove_device + TactileVectorToFingerPower.

Attributes:

gain	Force-to-power gain [1/N] applied after the deadband.
deadband	Contact force [N] below which no vibration is produced.
smoothing	EMA new-sample weight in [0, 1] (1.0 = no smoothing) applied to each finger power.
left_sensor_name	Scene entity name of the left-hand ContactSensor.
right_sensor_name	Scene entity name of the right-hand ContactSensor.
num_taxels	1 for a single rumble motor, one channel per finger for a glove.
saturation	Upper clamp on the normalized output amplitude in [0, 1].
collection_id	Push-tensor collection id pairing Isaac Teleop with the glove plugin process (the Manus plugin's default).
finger_order	Per-channel finger substrings, in glove channel order, matched against the contact sensor's body names to group each finger's links into one channel.

Methods:

__init__([left_sensor_name, ...])
make_signal_fn()	Return the signal source (environment -> per-hand vector) for this config.
build_sink(force_inputs, tracker_provider)	Build the isaacteleop HapticSink that renders the cached vectors.

gain: float

Force-to-power gain [1/N] applied after the deadband. Default maps ~10 N to full power.

deadband: float

Contact force [N] below which no vibration is produced.

smoothing: float

EMA new-sample weight in [0, 1] (1.0 = no smoothing) applied to each finger power.

__init__(left_sensor_name: str = <factory>, right_sensor_name: str = <factory>, num_taxels: int = <factory>, gain: float = <factory>, deadband: float = <factory>, saturation: float = <factory>, smoothing: float = <factory>, collection_id: str = <factory>, finger_order: list[str] = <factory>) → None

left_sensor_name: str

Scene entity name of the left-hand ContactSensor.

right_sensor_name: str

Scene entity name of the right-hand ContactSensor.

num_taxels: int

1 for a single rumble motor, one channel per finger for a glove.

Type:

Length of the per-hand output vector

saturation: float

Upper clamp on the normalized output amplitude in [0, 1].

collection_id: str

Push-tensor collection id pairing Isaac Teleop with the glove plugin process (the Manus plugin’s default). Change it to target a different glove vendor.

finger_order: list[str]

Per-channel finger substrings, in glove channel order, matched against the contact sensor’s body names to group each finger’s links into one channel. This is the sole source of the finger-channel count (num_taxels).

make_signal_fn() → SignalFn

Return the signal source (environment -> per-hand vector) for this config.

build_sink(force_inputs: dict[str, Any], tracker_provider: Callable[[], Any]) → tuple[Any, Any]

Build the isaacteleop HapticSink that renders the cached vectors.

Parameters:

• force_inputs – Per-endpoint graph output nodes, each carrying a TactileVector(num_taxels) fed from the driver every step.

• tracker_provider – Callable returning a DeviceIO tracker to reuse for devices that need one (e.g. a controller). Cross-process devices (e.g. a glove) may ignore it.

Returns:

A (connected_sink, tracker) tuple. connected_sink is the HapticSink.connect(...) result for TeleopSessionConfig(sinks=[...]) (a subgraph, not the sink node). tracker is the device’s DeviceIO tracker (or None); the session uses it to request the device’s OpenXR extensions (e.g. a glove’s XR_NVX1_push_tensor). It is returned separately because the connected subgraph does not expose the device.

class isaaclab_teleop.HapticFeedbackReceiver

Protocol for a teleop device that can render haptic feedback.

Deliberately a single-method protocol rather than an addition to DeviceBase: input-only devices (keyboard, space-mouse, gamepad) cannot honor a feedback call, so forcing the method onto every device would break Liskov substitutability. Scripts guard with isinstance(device, HapticFeedbackReceiver) and only drive feedback when the active device actually supports it.

Methods:

send_haptic(endpoint, values)	Render one frame of haptic output on a device endpoint.
__init__(*args, **kwargs)

send_haptic(endpoint: str, values: Sequence[float]) → None

Render one frame of haptic output on a device endpoint.

Parameters:

• endpoint – Which hand to actuate – ENDPOINT_LEFT or ENDPOINT_RIGHT.

• values – The per-hand output vector (length num_taxels): a single scalar for a rumble motor, one value per finger for a glove. An all-zero vector stops any active feedback. The signal-to-device mapping (gain, deadband, saturation) is applied downstream.

__init__(*args, **kwargs)

class isaaclab_teleop.HapticFeedbackDriver

Reads the per-hand signal from the scene and pushes it to the device.

One driver is shared by every teleop script. Construct it with create_haptic_feedback_driver() (which returns None when the env has no haptic config or the device cannot render it), then call update() once per step after env.step so the sensors hold fresh post-physics data.

Methods:

__init__(env, device, cfg)	Initialize the driver.
update()	Read the per-hand signal and forward each endpoint's vector to the device.
stop()	Zero every endpoint so any active feedback stops.

__init__(env: ManagerBasedRLEnv, device: HapticFeedbackReceiver, cfg: HapticFeedbackCfg)

Initialize the driver.

Parameters:

• env – The (unwrapped) environment whose scene owns the sensors.

• device – The teleop device implementing HapticFeedbackReceiver.

• cfg – Haptic feedback configuration providing the signal source.

update() → None

Read the per-hand signal and forward each endpoint’s vector to the device.

stop() → None

Zero every endpoint so any active feedback stops.

Call this on every frame the robot is not being stepped (teleop paused or session not started). Because the device re-emits the cached vector each frame, forgetting to zero it would leave the device rendering a stale grip force until the next reset.

isaaclab_teleop.create_haptic_feedback_driver(env: ManagerBasedRLEnv, device: object, env_cfg: object) → HapticFeedbackDriver | None

Build a HapticFeedbackDriver when the env and device support haptics.

Follows the same capability-discovery idiom the teleop scripts use for isaac_teleop: a haptic_feedback attribute on the env config plus a device that satisfies HapticFeedbackReceiver.

Parameters:

• env – The (unwrapped) environment.

• device – The active teleop device (any type).

• env_cfg – The environment configuration; checked for a haptic_feedback HapticFeedbackCfg attribute.

Returns:

A ready driver, or None if haptics are not configured or the device cannot render them.

XR Anchor

class isaaclab_teleop.XrAnchorSynchronizer

Keeps the XR anchor prim aligned with a reference prim according to XR config.

Methods:

__init__(xr_core, xr_cfg, xr_anchor_headset_path)
get_world_transform()	Return the anchor world transform.
sync_headset_to_anchor()	Sync XR anchor pose in USD for both dynamic and static anchoring.

__init__(xr_core: Any, xr_cfg: Any, xr_anchor_headset_path: str)

get_world_transform() → tuple[ndarray, ndarray] | None

Return the anchor world transform.

Returns the cached world transform that was computed by the most recent call to sync_headset_to_anchor(). Using the cached value avoids a Fabric/USD layer mismatch: when the XR anchor prim is a child of a physics-driven prim (e.g. the robot pelvis), reading GetFabricHierarchyWorldMatrixAttr would compose the Fabric-side parent transform (updated by physics) with a local xform that was decomposed against the USD-side parent (which can lag behind), producing an incorrect world matrix that drifts as the robot moves.

Returns:

A (position, quat_xyzw) tuple of numpy float64 arrays, or None if sync_headset_to_anchor() has not run yet.

sync_headset_to_anchor()

Sync XR anchor pose in USD for both dynamic and static anchoring.

For dynamic anchoring (anchor_prim_path is set), the reference prim’s world position is read from Fabric and anchor_pos is added as an offset. For static anchoring (no prim path), anchor_pos is used directly as the world position.

In both cases the function calls set_world_transform_matrix on the XR core so that the rendering anchor and the pipeline’s world_T_anchor matrix are guaranteed to agree, and caches the world transform for get_world_transform().

isaaclab_teleop.remove_camera_configs(env_cfg: Any) → Any

Removes cameras from environments when using XR devices.

Having additional cameras cause operation performance issues. This function scans the environment configuration for camera objects and removes them, along with any associated observation terms that reference these cameras.

Parameters:

env_cfg – The environment configuration to modify.

Returns:

The modified environment configuration with cameras removed.