Running Existing Scripts#


The main core interface extension in Isaac Lab omni.isaac.lab provides the main modules for actuators, objects, robots and sensors. We provide a list of demo scripts and tutorials. These showcase how to use the provided interfaces within a code in a minimal way.

A few quick showroom scripts to run and checkout:

  • Spawn different quadrupeds and make robots stand using position commands:

    ./ -p source/standalone/demos/
  • Spawn different arms and apply random joint position commands:

    ./ -p source/standalone/demos/
  • Spawn different hands and command them to open and close:

    ./ -p source/standalone/demos/
  • Spawn procedurally generated terrains with different configurations:

    ./ -p source/standalone/demos/
  • Spawn different deformable (soft) bodies and let them fall from a height:

    ./ -p source/standalone/demos/
  • Spawn multiple markers that are useful for visualizations:

    ./ -p source/standalone/demos/


With Isaac Lab, we also provide a suite of benchmark environments included in the omni.isaac.lab_tasks extension. We use the OpenAI Gym registry to register these environments. For each environment, we provide a default configuration file that defines the scene, observations, rewards and action spaces.

The list of environments available registered with OpenAI Gym can be found by running:

./ -p source/standalone/environments/

Basic agents#

These include basic agents that output zero or random agents. They are useful to ensure that the environments are configured correctly.

  • Zero-action agent on the Cart-pole example

    ./ -p source/standalone/environments/ --task Isaac-Cartpole-v0 --num_envs 32
  • Random-action agent on the Cart-pole example:

    ./ -p source/standalone/environments/ --task Isaac-Cartpole-v0 --num_envs 32

State machine#

We include examples on hand-crafted state machines for the environments. These help in understanding the environment and how to use the provided interfaces. The state machines are written in warp which allows efficient execution for large number of environments using CUDA kernels.

./ -p source/standalone/environments/state_machine/ --num_envs 32


We provide interfaces for providing commands in SE(2) and SE(3) space for robot control. In case of SE(2) teleoperation, the returned command is the linear x-y velocity and yaw rate, while in SE(3), the returned command is a 6-D vector representing the change in pose.

To play inverse kinematics (IK) control with a keyboard device:

./ -p source/standalone/environments/teleoperation/ --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --device keyboard

The script prints the teleoperation events configured. For keyboard, these are as follows:

Keyboard Controller for SE(3): Se3Keyboard
    Reset all commands: L
    Toggle gripper (open/close): K
    Move arm along x-axis: W/S
    Move arm along y-axis: A/D
    Move arm along z-axis: Q/E
    Rotate arm along x-axis: Z/X
    Rotate arm along y-axis: T/G
    Rotate arm along z-axis: C/V

Imitation Learning#

Using the teleoperation devices, it is also possible to collect data for learning from demonstrations (LfD). For this, we support the learning framework Robomimic (Linux only) and allow saving data in HDF5 format.

  1. Collect demonstrations with teleoperation for the environment Isaac-Lift-Cube-Franka-IK-Rel-v0:

    # step a: collect data with keyboard
    ./ -p source/standalone/workflows/robomimic/ --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --num_envs 1 --num_demos 10 --device keyboard
    # step b: inspect the collected dataset
    ./ -p source/standalone/workflows/robomimic/tools/ logs/robomimic/Isaac-Lift-Cube-Franka-IK-Rel-v0/hdf_dataset.hdf5
  2. Split the dataset into train and validation set:

    # install the dependencies
    sudo apt install cmake build-essential
    # install python module (for robomimic)
    ./ -i robomimic
    # split data
    ./ -p source/standalone//workflows/robomimic/tools/ logs/robomimic/Isaac-Lift-Cube-Franka-IK-Rel-v0/hdf_dataset.hdf5 --ratio 0.2
  3. Train a BC agent for Isaac-Lift-Cube-Franka-IK-Rel-v0 with Robomimic:

    ./ -p source/standalone/workflows/robomimic/ --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --algo bc --dataset logs/robomimic/Isaac-Lift-Cube-Franka-IK-Rel-v0/hdf_dataset.hdf5
  4. Play the learned model to visualize results:

    ./ -p source/standalone/workflows/robomimic/ --task Isaac-Lift-Cube-Franka-IK-Rel-v0 --checkpoint /PATH/TO/model.pth

Reinforcement Learning#

We provide wrappers to different reinforcement libraries. These wrappers convert the data from the environments into the respective libraries function argument and return types.

  • Training an agent with Stable-Baselines3 on Isaac-Cartpole-v0:

    # install python module (for stable-baselines3)
    ./ -i sb3
    # run script for training
    # note: we enable cpu flag since SB3 doesn't optimize for GPU anyway
    ./ -p source/standalone/workflows/sb3/ --task Isaac-Cartpole-v0 --headless --cpu
    # run script for playing with 32 environments
    ./ -p source/standalone/workflows/sb3/ --task Isaac-Cartpole-v0 --num_envs 32 --checkpoint /PATH/TO/
  • Training an agent with SKRL on Isaac-Reach-Franka-v0:

    # install python module (for skrl)
    ./ -i skrl
    # run script for training
    ./ -p source/standalone/workflows/skrl/ --task Isaac-Reach-Franka-v0 --headless
    # run script for playing with 32 environments
    ./ -p source/standalone/workflows/skrl/ --task Isaac-Reach-Franka-v0 --num_envs 32 --checkpoint /PATH/TO/
    # install python module (for skrl)
    ./ -i skrl
    # install skrl dependencies for JAX. Visit for more details
    ./ -p -m pip install skrl["jax"]
    # run script for training
    ./ -p source/standalone/workflows/skrl/ --task Isaac-Reach-Franka-v0 --headless --ml_framework jax
    # run script for playing with 32 environments
    ./ -p source/standalone/workflows/skrl/ --task Isaac-Reach-Franka-v0 --num_envs 32  --ml_framework jax --checkpoint /PATH/TO/
  • Training an agent with RL-Games on Isaac-Ant-v0:

    # install python module (for rl-games)
    ./ -i rl_games
    # run script for training
    ./ -p source/standalone/workflows/rl_games/ --task Isaac-Ant-v0 --headless
    # run script for playing with 32 environments
    ./ -p source/standalone/workflows/rl_games/ --task Isaac-Ant-v0 --num_envs 32 --checkpoint /PATH/TO/model.pth
  • Training an agent with RSL-RL on Isaac-Reach-Franka-v0:

    # install python module (for rsl-rl)
    ./ -i rsl_rl
    # run script for training
    ./ -p source/standalone/workflows/rsl_rl/ --task Isaac-Reach-Franka-v0 --headless
    # run script for playing with 32 environments
    ./ -p source/standalone/workflows/rsl_rl/ --task Isaac-Reach-Franka-v0 --num_envs 32 --load_run run_folder_name --checkpoint

All the scripts above log the training progress to Tensorboard in the logs directory in the root of the repository. The logs directory follows the pattern logs/<library>/<task>/<date-time>, where <library> is the name of the learning framework, <task> is the task name, and <date-time> is the timestamp at which the training script was executed.

To view the logs, run:

# execute from the root directory of the repository
./ -p -m tensorboard.main --logdir=logs