How-to Guides

Find step-by-step guides for building with Isaac Lab. Search by topic, then open a guide without leaving this page.

Search guides

Simulation Fundamentals

Creating an empty scene

Launch an empty simulation and learn the core startup sequence.

Spawning prims into the scene

Add lights, ground planes, and primitive shapes to a simulation stage.

Deep-dive into AppLauncher

Configure and launch simulation applications from Python and the command line.

Assets

Adding a new robot to Isaac Lab

Bring a robot asset into Isaac Lab and define its articulation configuration.

Interacting with a rigid object

Create, reset, and command a rigid object through the simulation API.

Interacting with an articulation

Work with joint state, commands, and articulation data.

Interacting with a deformable object

Spawn and manipulate deformable bodies in a scene.

Interacting with a surface gripper

Attach and release rigid objects with a surface gripper.

Importing a new asset

Convert URDF, MJCF, or mesh assets into USD for use in Isaac Lab.

Writing an asset configuration

Turn an imported robot into a reusable articulation configuration.

Robot configurations

Understand the structure and conventions of supported robot configurations.

Making a physics prim fixed

Convert a floating asset into a fixed object in the simulation.

Spawning multiple assets

Batch rigid objects and vary asset configurations across environments.

Preparing an asset for Newton with MJWarp

Prepare an asset and task to run with the Newton MJWarp physics preset.

Scenes and Cloning

Using the interactive scene

Compose assets and sensors with the higher-level interactive scene interface.

Cloning environments

Choose cloning strategies, build heterogeneous scenes, and filter collisions.

Environments and Training

Creating a manager-based base environment

Build a non-RL environment from reusable manager terms.

Creating a manager-based RL environment

Add rewards, terminations, curricula, and commands for reinforcement learning.

Creating a direct workflow RL environment

Implement an RL task with direct control over the environment loop.

Registering an environment

Register an Isaac Lab task with Gymnasium and expose its configurations.

Training with an RL agent

Launch training and inference with a supported reinforcement learning library.

Configuring an RL agent

Customize agent settings and training hyperparameters.

Modifying an existing direct RL environment

Extend and adjust a direct workflow task without rebuilding it from scratch.

Policy inference in a USD environment

Run a trained policy against an environment defined in a USD stage.

Wrapping environments

Adapt Isaac Lab environments to external reinforcement learning interfaces.

Adding your own learning library

Integrate an additional learning framework with Isaac Lab tasks.

Running scripted state machines

Drive environments with deterministic state-machine policies.

Curriculum utilities

Change environment parameters dynamically during training.

Transferring policies between PhysX and Newton

Validate and evaluate policy checkpoints across the PhysX and Newton backends.

Sensors, Cameras, and Rendering

Adding sensors on a robot

Add camera, ray-caster, and contact sensors to an environment.

Saving rendered images and 3D re-projection

Save camera outputs and reconstruct point clouds from depth images.

Finding how many cameras to train with

Estimate camera throughput and memory limits for your hardware.

Configuring RTX rendering settings

Tune RTX rendering quality and performance options.

Capturing sensor frames during training

Record selected sensor outputs from a running training job.

Controllers

Using a task-space controller

Control a robot end effector with differential inverse kinematics.

Using an operational space controller

Apply operational-space control to a robot manipulator.

Simulation and Data

Working with ProxyArray

Consume asset and sensor data through Torch and Warp while keeping retained views valid.

Simulation performance

Diagnose bottlenecks and improve simulation throughput.

Teleoperation

Setting up Isaac Teleop with CloudXR

Connect XR devices through CloudXR for immersive teleoperation.

Setting up Haply teleoperation

Use Haply devices for robot control with directional force feedback.

Tools and Workflows

Recording animations of simulations

Capture simulation state and export an animation.

Mastering Omniverse for robotics

Find Omniverse workflows and resources relevant to Isaac Lab.

Profiling Isaac Lab with Nsight Systems

Capture and inspect runtime traces with NVIDIA Nsight Systems.

Note

This collection is a work in progress. If a question is not answered here, open an issue on the Isaac Lab GitHub repository.