isaaclab_experimental.envs#

Sub-package for environment definitions.

Environments define the interface between the agent and the simulation. In the simplest case, the environment provides the agent with the current observations and executes the actions provided by the agent. However, the environment can also provide additional information such as the current reward, done flag, and information about the current episode.

There are two types of environment designing workflows:

  • Manager-based: The environment is decomposed into individual components (or managers) for different aspects (such as computing observations, applying actions, and applying randomization. The users mainly configure the managers and the environment coordinates the managers and calls their functions.

  • Direct: The user implements all the necessary functionality directly into a single class directly without the need for additional managers.

Based on these workflows, there are the following environment classes for single and multi-agent RL:

Single-Agent RL:

  • ManagerBasedEnv: The manager-based workflow base environment which only provides the agent with the current observations and executes the actions provided by the agent.

  • ManagerBasedRLEnv: The manager-based workflow RL task environment which besides the functionality of the base environment also provides additional Markov Decision Process (MDP) related information such as the current reward, done flag, and information.

  • DirectRLEnv: The direct workflow RL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

Multi-Agent RL (MARL):

  • DirectMARLEnv: The direct workflow MARL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

For more information about the workflow design patterns, see the Task Design Workflows section.

Additional Public Classes#

The following classes are part of the public isaaclab_experimental.envs API.

DirectRLEnvWarp

The superclass for the direct workflow to design environments.

InteractiveSceneData

Live views of the asset data of an InteractiveScene, by scene entity name and field.

ManagerBasedEnvWarp

The base environment for the manager-based workflow (experimental fork).

ManagerBasedRLEnvWarp

The superclass for the manager-based workflow reinforcement learning-based environments.

MDP2Env

An interactive scene driven by an MDP2 task, like a ManagerBasedEnv without managers.

MDP2EnvCfg

Configuration for MDP2Env.

MDP2RLEnv

A Gym environment driven by an MDP2 task, like a ManagerBasedRLEnv without managers.

MDP2RLEnvCfg

Configuration for MDP2RLEnv.

ResetStep

Built-in work of a reset, in MDP2EnvCfg.reset_step.

Stage

User work in a control step or a reset, at its place in MDP2EnvCfg.step or MDP2EnvCfg.reset_step, such as kernels that update a scene field the MDP reads.

Step

Built-in work of a control step, in MDP2EnvCfg.step.

class isaaclab_experimental.envs.DirectRLEnvWarp[source]#

Bases: DirectRLEnv

The superclass for the direct workflow to design environments.

This class implements the core functionality for reinforcement learning (RL) environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments.

While the environment itself is implemented as a vectorized environment, we do not inherit from gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use the gym.Env over here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.

Note

For vectorized environments, it is recommended to only call the reset() method once before the first call to step(), i.e. after the environment is created. After that, the step() function handles the reset of terminated sub-environments. in a vectorized environment.

Methods:

__init__(cfg[, render_mode])

Initialize the environment.

__new__(cls, *args, **kwargs)

__init__(cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs)[source]#

Initialize the environment.

Parameters:
  • cfg – The configuration object for the environment.

  • render_mode – The render mode for the environment. Defaults to None, which is similar to "human".

Raises:

RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.

static __new__(cls, *args: Any, **kwargs: Any) → Any#
class isaaclab_experimental.envs.InteractiveSceneData[source]#

Live views of the asset data of an InteractiveScene, by scene entity name and field.

Fields keep the names, units, and frames of the asset data: positions in m, orientations as (x, y, z, w) quaternions, joint values in rad or m for revolute or prismatic joints, and times in s. Articulations and rigid objects provide their root state:

  • root_link_pose_w, default_root_pose: wp.transformf per world, in the world frame and in the frame of the world’s environment origin respectively;

  • root_com_vel_w, default_root_vel: wp.spatial_vectorf per world, linear [m/s] then angular [rad/s], in the world frame;

  • body_link_pose_w: wp.transformf per world and body, in the world frame.

Articulations also provide joint_pos, joint_vel, joint_acc [rad/s^2 or m/s^2], default_joint_pos, default_joint_vel, joint_pos_target, joint_effort_target and applied_effort [N·m or N], soft_joint_vel_limits, and soft_joint_pos_limits (wp.vec2f), per world and joint. The scene entity provides env_origins [m] and gravity_w (wp.vec3f, with magnitude) [m/s^2] per world.

Contact sensors provide net_forces_w [N] and their histories net_forces_w_history and net_normal_forces_w_history (wp.vec3f, most recent first), and the air and contact times current_air_time, last_air_time, current_contact_time, and last_contact_time, per world and body. Sensors with filter objects also provide normal_force_matrix_w (wp.vec3f) [N] per world, body, and filter.

Events and actions write fields in place, and commit() makes the simulation see the writes to the joint and root state. Where the asset’s data is the solver’s own state, as Newton’s joint positions and root state usually are, a commit only tells the solver which worlds changed. Otherwise, as for the joint positions of Newton articulations with ball joints or on other physics backends, it passes the written worlds through the asset’s write_*_to_sim_mask methods; joint velocities always go through them, since the asset keeps the previous ones for joint accelerations. The default state and the joint targets are plain buffers that need no commit. gravity_w is writable where it is the physics model’s own (Newton). Derived and sensor fields are read-only.

Methods:

__init__(scene, sim, step_dt)

data(entity, field)

Return a live view of an entity field, of shape (num_envs, ...).

entity(name)

Return the scene object of an entity, such as an articulation.

commit(mask, fields)

Pass the written fields of the masked worlds to the simulation, and recompute their body poses.

__init__(scene: InteractiveScene, sim: SimulationContext, step_dt: float) → None[source]#
data(entity: str, field: str) → ProxyArray[source]#

Return a live view of an entity field, of shape (num_envs, ...).

entity(name: str) → Any[source]#

Return the scene object of an entity, such as an articulation.

commit(mask: array, fields: Collection[FieldSelector]) → None[source]#

Pass the written fields of the masked worlds to the simulation, and recompute their body poses. Safe to capture in a CUDA graph.

Only the assets whose state was written are recomputed, so a commit costs the fields it passes on.

class isaaclab_experimental.envs.ManagerBasedEnvWarp[source]#

Bases: object

The base environment for the manager-based workflow (experimental fork).

The implementation mirrors isaaclab.envs.ManagerBasedEnv to provide an isolated base class for experimental Warp-based workflows.

Methods:

__init__(cfg)

Initialize the environment.

__new__(*args, **kwargs)

__init__(cfg: ManagerBasedEnvCfg)[source]#

Initialize the environment.

Parameters:

cfg – The configuration object for the environment.

Raises:

RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.

classmethod __new__(*args, **kwargs)#
class isaaclab_experimental.envs.ManagerBasedRLEnvWarp[source]#

Bases: ManagerBasedEnvWarp, Env

The superclass for the manager-based workflow reinforcement learning-based environments.

This class inherits from ManagerBasedEnvWarp and implements the core functionality for reinforcement learning-based environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments. The number of sub-environments is specified using the num_envs.

Each observation from the environment is a batch of observations for each sub- environments. The method step() is also expected to receive a batch of actions for each sub-environment.

While the environment itself is implemented as a vectorized environment, we do not inherit from gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use the gym.Env over here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.

Note

For vectorized environments, it is recommended to only call the reset() method once before the first call to step(), i.e. after the environment is created. After that, the step() function handles the reset of terminated sub-environments. This is because the simulator does not support resetting individual sub-environments in a vectorized environment.

Methods:

__init__(cfg[, render_mode])

Initialize the environment.

__new__(cls, *args, **kwargs)

__init__(cfg: ManagerBasedRLEnvCfg, render_mode: str | None = None, **kwargs)[source]#

Initialize the environment.

Parameters:
  • cfg – The configuration for the environment.

  • render_mode – The render mode for the environment. Defaults to None, which is similar to "human".

static __new__(cls, *args: Any, **kwargs: Any) → Any#
class isaaclab_experimental.envs.MDP2Env[source]#

Bases: object

An interactive scene driven by an MDP2 task, like a ManagerBasedEnv without managers.

A control step applies the actions, steps the physics decimation times, evaluates the MDP, resets the worlds whose episodes ended, and observes them, in the order of the config’s step and reset_step, with their Stage s. Resets are masks over the worlds, so every step after the first replays as CUDA graphs, physics included: one graph for a fully capturable step, and otherwise one per run of capturable work, with the work that cannot be captured run between them. Newton only.

Methods:

__init__(cfg)

__new__(*args, **kwargs)

__init__(cfg: MDP2EnvCfg) → None[source]#
classmethod __new__(*args, **kwargs)#
class isaaclab_experimental.envs.MDP2EnvCfg[source]#

Bases: object

Configuration for MDP2Env.

Methods:

__new__(*args, **kwargs)

__init__([sim, scene, task, task_worlds, ...])

classmethod __new__(*args, **kwargs)#
__init__(sim: ~isaaclab.sim.simulation_cfg.SimulationCfg = <factory>, scene: ~isaaclab.scene.interactive_scene_cfg.InteractiveSceneCfg = <factory>, task: ~isaaclab_experimental.mdp2.config.TaskCfg | list[~isaaclab_experimental.mdp2.config.TaskCfg] = <factory>, task_worlds: list[int] | None = <factory>, decimation: int = <factory>, backend: ~isaaclab_experimental.mdp2.compiled.Backend = <factory>, seed: int | None = <factory>, capture: bool = <factory>, step: list[~isaaclab_experimental.envs.mdp2_env_cfg.Step | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, reset_step: list[~isaaclab_experimental.envs.mdp2_env_cfg.ResetStep | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, verify_capture: int = <factory>) → None#
class isaaclab_experimental.envs.MDP2RLEnv[source]#

Bases: MDP2Env, Env

A Gym environment driven by an MDP2 task, like a ManagerBasedRLEnv without managers.

step() returns copies of the observations after resets, and the rewards and the terminated and truncated flags of the step as Torch views of buffers that the next step overwrites. A world is truncated only if it did not terminate. extras["log"] holds, per term, the average over worlds of their last finished episode: the reward sum per second, whether the term ended it, and the value that each logged quantity had at its end. With compute_final_obs, extras["final_obs"] holds the observations that the worlds that ended had before their reset, also as views; only the rows of the worlds that ended are of this step.

Methods:

__init__(cfg[, render_mode])

__new__(cls, *args, **kwargs)

__init__(cfg: MDP2RLEnvCfg, render_mode: str | None = None, **kwargs: Any) → None[source]#
static __new__(cls, *args: Any, **kwargs: Any) → Any#
class isaaclab_experimental.envs.MDP2RLEnvCfg[source]#

Bases: MDP2EnvCfg

Configuration for MDP2RLEnv.

Methods:

__new__(*args, **kwargs)

__init__([sim, scene, task, task_worlds, ...])

classmethod __new__(*args, **kwargs)#
__init__(sim: ~isaaclab.sim.simulation_cfg.SimulationCfg = <factory>, scene: ~isaaclab.scene.interactive_scene_cfg.InteractiveSceneCfg = <factory>, task: ~isaaclab_experimental.mdp2.config.TaskCfg | list[~isaaclab_experimental.mdp2.config.TaskCfg] = <factory>, task_worlds: list[int] | None = <factory>, decimation: int = <factory>, backend: ~isaaclab_experimental.mdp2.compiled.Backend = <factory>, seed: int | None = <factory>, capture: bool = <factory>, step: list[~isaaclab_experimental.envs.mdp2_env_cfg.Step | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, reset_step: list[~isaaclab_experimental.envs.mdp2_env_cfg.ResetStep | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, verify_capture: int = <factory>, is_finite_horizon: bool = <factory>, compute_final_obs: bool = <factory>, log_dir: str | None = <factory>, video_recorders: list = <factory>) → None#
class isaaclab_experimental.envs.ResetStep[source]#

Built-in work of a reset, in MDP2EnvCfg.reset_step.

class isaaclab_experimental.envs.Stage[source]#

User work in a control step or a reset, at its place in MDP2EnvCfg.step or MDP2EnvCfg.reset_step, such as kernels that update a scene field the MDP reads.

fn(env) may read and write anything, through the environment, its scene, and its MDP state. A capturable stage runs inside the step’s CUDA graph: it is called once, at capture, and its launches replay every step, so it must not read values on the host or draw from Torch’s random generators, and Python branches take the path they took at capture. Check it with MDP2EnvCfg.verify_capture. A stage that is not capturable runs on the host between the graphs. After each stage, the simulation applies the model and state changes it made through the assets.

Methods:

__new__(*args, **kwargs)

__init__(fn[, capturable])

classmethod __new__(*args, **kwargs)#
__init__(fn: Callable[[Any], None], capturable: bool = True) → None#
class isaaclab_experimental.envs.Step[source]#

Built-in work of a control step, in MDP2EnvCfg.step.