isaaclab_experimental.envs

Sub-package for environment definitions.

Environments define the interface between the agent and the simulation. In the simplest case, the environment provides the agent with the current observations and executes the actions provided by the agent. However, the environment can also provide additional information such as the current reward, done flag, and information about the current episode.

There are two types of environment designing workflows:

• Manager-based: The environment is decomposed into individual components (or managers) for different aspects (such as computing observations, applying actions, and applying randomization. The users mainly configure the managers and the environment coordinates the managers and calls their functions.

• Direct: The user implements all the necessary functionality directly into a single class directly without the need for additional managers.

Based on these workflows, there are the following environment classes for single and multi-agent RL:

Single-Agent RL:

• ManagerBasedEnv: The manager-based workflow base environment which only provides the agent with the current observations and executes the actions provided by the agent.

• ManagerBasedRLEnv: The manager-based workflow RL task environment which besides the functionality of the base environment also provides additional Markov Decision Process (MDP) related information such as the current reward, done flag, and information.

• DirectRLEnv: The direct workflow RL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

Multi-Agent RL (MARL):

• DirectMARLEnv: The direct workflow MARL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

For more information about the workflow design patterns, see the Task Design Workflows section.

Additional Public Classes

The following classes are part of the public isaaclab_experimental.envs API.

DirectRLEnvWarp	The superclass for the direct workflow to design environments.
InteractiveSceneWarp	Interactive scene with warp-native env_mask support for reset.
ManagerBasedEnvWarp	The base environment for the manager-based workflow (experimental fork).
ManagerBasedRLEnvWarp	The superclass for the manager-based workflow reinforcement learning-based environments.

class isaaclab_experimental.envs.DirectRLEnvWarp

Bases: DirectRLEnv

The superclass for the direct workflow to design environments.

This class implements the core functionality for reinforcement learning (RL) environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments.

While the environment itself is implemented as a vectorized environment, we do not inherit from gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use the gym.Env over here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.

Note

For vectorized environments, it is recommended to only call the reset() method once before the first call to step(), i.e. after the environment is created. After that, the step() function handles the reset of terminated sub-environments. in a vectorized environment.

Methods:

__init__(cfg[, render_mode])	Initialize the environment.
__new__(cls, *args, **kwargs)

__init__(cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs)

Initialize the environment.

Parameters:

• cfg – The configuration object for the environment.

• render_mode – The render mode for the environment. Defaults to None, which is similar to "human".

Raises:

RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.

static __new__(cls, *args: Any, **kwargs: Any) → Any

class isaaclab_experimental.envs.InteractiveSceneWarp

Bases: InteractiveScene

Interactive scene with warp-native env_mask support for reset.

Extends InteractiveScene to accept a boolean warp mask for selective resets, avoiding the need to convert between env_ids and masks.

Methods:

__new__(*args, **kwargs)
__init__(cfg)	Initializes the scene.

classmethod __new__(*args, **kwargs)

__init__(cfg: InteractiveSceneCfg)

Initializes the scene.

Parameters:

cfg – The configuration class for the scene.

class isaaclab_experimental.envs.ManagerBasedEnvWarp

Bases: object

The base environment for the manager-based workflow (experimental fork).

The implementation mirrors isaaclab.envs.ManagerBasedEnv to provide an isolated base class for experimental Warp-based workflows.

Methods:

__init__(cfg)	Initialize the environment.
__new__(*args, **kwargs)

__init__(cfg: ManagerBasedEnvCfg)

Initialize the environment.

Parameters:

cfg – The configuration object for the environment.

Raises:

RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.

classmethod __new__(*args, **kwargs)

class isaaclab_experimental.envs.ManagerBasedRLEnvWarp

Bases: ManagerBasedEnvWarp, Env

The superclass for the manager-based workflow reinforcement learning-based environments.

This class inherits from ManagerBasedEnv and implements the core functionality for reinforcement learning-based environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments. The number of sub-environments is specified using the num_envs.

Each observation from the environment is a batch of observations for each sub- environments. The method step() is also expected to receive a batch of actions for each sub-environment.

While the environment itself is implemented as a vectorized environment, we do not inherit from gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use the gym.Env over here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.

Note

For vectorized environments, it is recommended to only call the reset() method once before the first call to step(), i.e. after the environment is created. After that, the step() function handles the reset of terminated sub-environments. This is because the simulator does not support resetting individual sub-environments in a vectorized environment.

Methods:

__init__(cfg[, render_mode])	Initialize the environment.
__new__(cls, *args, **kwargs)

__init__(cfg: ManagerBasedRLEnvCfg, render_mode: str | None = None, **kwargs)

Initialize the environment.

Parameters:

• cfg – The configuration for the environment.

• render_mode – The render mode for the environment. Defaults to None, which is similar to "human".

static __new__(cls, *args: Any, **kwargs: Any) → Any