Application Development#
Extensions#
Extensions are the recommended way to develop applications in Isaac Sim. They are
modularized packages that formulate the Omniverse ecosystem. Each extension
provides a set of functionalities that can be used by other extensions or
standalone applications. A folder is recognized as an extension if it contains
an extension.toml
file in the config
directory. More information on extensions can be found in the
Omniverse documentation.
Each extension in Isaac Lab is written as a python package and follows the following structure:
<extension-name>
├── config
│ └── extension.toml
├── docs
│ ├── CHANGELOG.md
│ └── README.md
├── <extension-name>
│ ├── __init__.py
│ ├── ....
│ └── scripts
├── setup.py
└── tests
The config/extension.toml
file contains the metadata of the extension. This
includes the name, version, description, dependencies, etc. This information is used
by Omniverse to load the extension. The docs
directory contains the documentation
for the extension with more detailed information about the extension and a CHANGELOG
file that contains the changes made to the extension in each version.
The <extension-name>
directory contains the main python package for the extension.
It may also contains the scripts
directory for keeping python-based applications
that are loaded into Omniverse when then extension is enabled using the
Extension Manager.
More specifically, when an extension is enabled, the python module specified in the
config/extension.toml
file is loaded and scripts that contains children of the
omni.ext.IExt
class are executed.
import omni.ext
class MyExt(omni.ext.IExt):
"""My extension application."""
def on_startup(self, ext_id):
"""Called when the extension is loaded."""
pass
def on_shutdown(self):
"""Called when the extension is unloaded.
It releases all references to the extension and cleans up any resources.
"""
pass
While loading extensions into Omniverse happens automatically, using the python package
in standalone applications requires additional steps. To simplify the build process and
avoiding the need to understand the premake
build system used by Omniverse, we directly use the setuptools
python package to build the python module provided by the extensions. This is done by the
setup.py
file in the extension directory.
Note
The setup.py
file is not required for extensions that are only loaded into Omniverse
using the Extension Manager.
Lastly, the tests
directory contains the unit tests for the extension. These are written
using the unittest framework. It is
important to note that Omniverse also provides a similar
testing framework.
However, it requires going through the build process and does not support testing of the python module in
standalone applications.
Custom Extension Dependency Management#
Certain extensions may have dependencies which require installation of additional packages before the extension
can be used. While Python dependencies are handled by the setuptools
package and specified in the setup.py
file, non-Python dependencies such as ROS
packages or apt packages are not handled by setuptools.
To handle these dependencies, we have created an additional setup procedure described in the next section.
There are two types of dependencies that can be specified in the extension.toml
file
under the isaac_lab_settings
section:
apt_deps: A list of apt packages that need to be installed. These are installed using the apt package manager.
ros_ws: The path to the ROS workspace that contains the ROS packages. These are installed using the rosdep dependency manager.
As an example, the following extension.toml
file specifies the dependencies for the extension:
[isaac_lab_settings]
# apt dependencies
apt_deps = ["libboost-all-dev"]
# ROS workspace
# note: if this path is relative, it is relative to the extension directory's root
ros_ws = "/home/user/catkin_ws"
These dependencies are installed using the install_deps.py
script provided in the tools
directory.
To install all dependencies for all extensions, run the following command:
# execute from the root of the repository
# the script expects the type of dependencies to install and the path to the extensions directory
# available types are: 'apt', 'rosdep' and 'all'
python tools/install_deps.py all ${ISAACLAB_PATH}/source/extensions
Note
Currently, this script is automatically executed during the build process of the Dockerfile.base
and Dockerfile.ros2
. This ensures that all the ‘apt’ and ‘rosdep’ dependencies are installed
before building the extensions respectively.
Standalone applications#
In a typical Omniverse workflow, the simulator is launched first, after which the extensions are enabled that load the python module and run the python application. While this is a recommended workflow, it is not always possible to use this workflow.
For example, for robot learning, it is essential to have complete control over simulation stepping
and all the other functionalities instead of asynchronously waiting for the simulator to step. In
such cases, it is necessary to write a standalone application that launches the simulator using
AppLauncher
and allows complete control over the simulation through
the SimulationContext
class.
The following snippet shows how to write a standalone application:
"""Launch Isaac Sim Simulator first."""
from omni.isaac.lab.app import AppLauncher
# launch omniverse app
app_launcher = AppLauncher(headless=False)
simulation_app = app_launcher.app
"""Rest everything follows."""
from omni.isaac.lab.sim import SimulationContext
if __name__ == "__main__":
# get simulation context
simulation_context = SimulationContext()
# reset and play simulation
simulation_context.reset()
# step simulation
simulation_context.step()
# stop simulation
simulation_context.stop()
# close the simulation
simulation_app.close()
It is necessary to launch the simulator before running any other code because extensions are hot-loaded when the simulator starts. Many Omniverse modules become available only after the simulator is launched. To do this, use the :class:~omni.isaac.lab.app.AppLauncher class to start the simulator. After that, the :class:~omni.isaac.lab.sim.SimulationContext class can be used to control the simulation. For further details, we recommend exploring the Isaac Lab tutorials.