isaaclab_experimental.envs#
Sub-package for environment definitions.
Environments define the interface between the agent and the simulation. In the simplest case, the environment provides the agent with the current observations and executes the actions provided by the agent. However, the environment can also provide additional information such as the current reward, done flag, and information about the current episode.
There are two types of environment designing workflows:
Manager-based: The environment is decomposed into individual components (or managers) for different aspects (such as computing observations, applying actions, and applying randomization. The users mainly configure the managers and the environment coordinates the managers and calls their functions.
Direct: The user implements all the necessary functionality directly into a single class directly without the need for additional managers.
Based on these workflows, there are the following environment classes for single and multi-agent RL:
Single-Agent RL:
ManagerBasedEnv: The manager-based workflow base environment which only provides the agent with the current observations and executes the actions provided by the agent.ManagerBasedRLEnv: The manager-based workflow RL task environment which besides the functionality of the base environment also provides additional Markov Decision Process (MDP) related information such as the current reward, done flag, and information.DirectRLEnv: The direct workflow RL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.
Multi-Agent RL (MARL):
DirectMARLEnv: The direct workflow MARL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.
For more information about the workflow design patterns, see the Task Design Workflows section.
Additional Public Classes#
The following classes are part of the public isaaclab_experimental.envs API.
The superclass for the direct workflow to design environments. |
|
Live views of the asset data of an |
|
The base environment for the manager-based workflow (experimental fork). |
|
The superclass for the manager-based workflow reinforcement learning-based environments. |
|
An interactive scene driven by an MDP2 task, like a |
|
Configuration for |
|
A Gym environment driven by an MDP2 task, like a |
|
Configuration for |
|
Built-in work of a reset, in |
|
User work in a control step or a reset, at its place in |
|
Built-in work of a control step, in |
- class isaaclab_experimental.envs.DirectRLEnvWarp[source]#
Bases:
DirectRLEnvThe superclass for the direct workflow to design environments.
This class implements the core functionality for reinforcement learning (RL) environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments.
While the environment itself is implemented as a vectorized environment, we do not inherit from
gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use thegym.Envover here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.Note
For vectorized environments, it is recommended to only call the
reset()method once before the first call tostep(), i.e. after the environment is created. After that, thestep()function handles the reset of terminated sub-environments. in a vectorized environment.Methods:
- __init__(cfg: DirectRLEnvCfg, render_mode: str | None = None, **kwargs)[source]#
Initialize the environment.
- Parameters:
- Raises:
RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.
- static __new__(cls, *args: Any, **kwargs: Any) Any#
- class isaaclab_experimental.envs.InteractiveSceneData[source]#
Live views of the asset data of an
InteractiveScene, by scene entity name and field.Fields keep the names, units, and frames of the asset data: positions in m, orientations as
(x, y, z, w)quaternions, joint values in rad or m for revolute or prismatic joints, and times in s. Articulations and rigid objects provide their root state:root_link_pose_w,default_root_pose:wp.transformfper world, in the world frame and in the frame of the world’s environment origin respectively;root_com_vel_w,default_root_vel:wp.spatial_vectorfper world, linear [m/s] then angular [rad/s], in the world frame;body_link_pose_w:wp.transformfper world and body, in the world frame.
Articulations also provide
joint_pos,joint_vel,joint_acc[rad/s^2 or m/s^2],default_joint_pos,default_joint_vel,joint_pos_target,joint_effort_targetandapplied_effort[N·m or N],soft_joint_vel_limits, andsoft_joint_pos_limits(wp.vec2f), per world and joint. Thesceneentity providesenv_origins[m] andgravity_w(wp.vec3f, with magnitude) [m/s^2] per world.Contact sensors provide
net_forces_w[N] and their historiesnet_forces_w_historyandnet_normal_forces_w_history(wp.vec3f, most recent first), and the air and contact timescurrent_air_time,last_air_time,current_contact_time, andlast_contact_time, per world and body. Sensors with filter objects also providenormal_force_matrix_w(wp.vec3f) [N] per world, body, and filter.Events and actions write fields in place, and
commit()makes the simulation see the writes to the joint and root state. Where the asset’s data is the solver’s own state, as Newton’s joint positions and root state usually are, a commit only tells the solver which worlds changed. Otherwise, as for the joint positions of Newton articulations with ball joints or on other physics backends, it passes the written worlds through the asset’swrite_*_to_sim_maskmethods; joint velocities always go through them, since the asset keeps the previous ones for joint accelerations. The default state and the joint targets are plain buffers that need no commit.gravity_wis writable where it is the physics model’s own (Newton). Derived and sensor fields are read-only.Methods:
__init__(scene, sim, step_dt)data(entity, field)Return a live view of an entity field, of shape
(num_envs, ...).entity(name)Return the scene object of an entity, such as an articulation.
commit(mask, fields)Pass the written fields of the masked worlds to the simulation, and recompute their body poses.
- __init__(scene: InteractiveScene, sim: SimulationContext, step_dt: float) None[source]#
- data(entity: str, field: str) ProxyArray[source]#
Return a live view of an entity field, of shape
(num_envs, ...).
- commit(mask: array, fields: Collection[FieldSelector]) None[source]#
Pass the written fields of the masked worlds to the simulation, and recompute their body poses. Safe to capture in a CUDA graph.
Only the assets whose state was written are recomputed, so a commit costs the fields it passes on.
- class isaaclab_experimental.envs.ManagerBasedEnvWarp[source]#
Bases:
objectThe base environment for the manager-based workflow (experimental fork).
The implementation mirrors
isaaclab.envs.ManagerBasedEnvto provide an isolated base class for experimental Warp-based workflows.Methods:
- __init__(cfg: ManagerBasedEnvCfg)[source]#
Initialize the environment.
- Parameters:
cfg¶ – The configuration object for the environment.
- Raises:
RuntimeError – If a simulation context already exists. The environment must always create one since it configures the simulation context and controls the simulation.
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.envs.ManagerBasedRLEnvWarp[source]#
Bases:
ManagerBasedEnvWarp,EnvThe superclass for the manager-based workflow reinforcement learning-based environments.
This class inherits from
ManagerBasedEnvWarpand implements the core functionality for reinforcement learning-based environments. It is designed to be used with any RL library. The class is designed to be used with vectorized environments, i.e., the environment is expected to be run in parallel with multiple sub-environments. The number of sub-environments is specified using thenum_envs.Each observation from the environment is a batch of observations for each sub- environments. The method
step()is also expected to receive a batch of actions for each sub-environment.While the environment itself is implemented as a vectorized environment, we do not inherit from
gym.vector.VectorEnv. This is mainly because the class adds various methods (for wait and asynchronous updates) which are not required. Additionally, each RL library typically has its own definition for a vectorized environment. Thus, to reduce complexity, we directly use thegym.Envover here and leave it up to library-defined wrappers to take care of wrapping this environment for their agents.Note
For vectorized environments, it is recommended to only call the
reset()method once before the first call tostep(), i.e. after the environment is created. After that, thestep()function handles the reset of terminated sub-environments. This is because the simulator does not support resetting individual sub-environments in a vectorized environment.Methods:
- __init__(cfg: ManagerBasedRLEnvCfg, render_mode: str | None = None, **kwargs)[source]#
Initialize the environment.
- static __new__(cls, *args: Any, **kwargs: Any) Any#
- class isaaclab_experimental.envs.MDP2Env[source]#
Bases:
objectAn interactive scene driven by an MDP2 task, like a
ManagerBasedEnvwithout managers.A control step applies the actions, steps the physics
decimationtimes, evaluates the MDP, resets the worlds whose episodes ended, and observes them, in the order of the config’sstepandreset_step, with theirStages. Resets are masks over the worlds, so every step after the first replays as CUDA graphs, physics included: one graph for a fully capturable step, and otherwise one per run of capturable work, with the work that cannot be captured run between them. Newton only.Methods:
- __init__(cfg: MDP2EnvCfg) None[source]#
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.envs.MDP2EnvCfg[source]#
Bases:
objectConfiguration for
MDP2Env.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(sim: ~isaaclab.sim.simulation_cfg.SimulationCfg = <factory>, scene: ~isaaclab.scene.interactive_scene_cfg.InteractiveSceneCfg = <factory>, task: ~isaaclab_experimental.mdp2.config.TaskCfg | list[~isaaclab_experimental.mdp2.config.TaskCfg] = <factory>, task_worlds: list[int] | None = <factory>, decimation: int = <factory>, backend: ~isaaclab_experimental.mdp2.compiled.Backend = <factory>, seed: int | None = <factory>, capture: bool = <factory>, step: list[~isaaclab_experimental.envs.mdp2_env_cfg.Step | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, reset_step: list[~isaaclab_experimental.envs.mdp2_env_cfg.ResetStep | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, verify_capture: int = <factory>) None#
- class isaaclab_experimental.envs.MDP2RLEnv[source]#
-
A Gym environment driven by an MDP2 task, like a
ManagerBasedRLEnvwithout managers.step()returns copies of the observations after resets, and the rewards and the terminated and truncated flags of the step as Torch views of buffers that the next step overwrites. A world is truncated only if it did not terminate.extras["log"]holds, per term, the average over worlds of their last finished episode: the reward sum per second, whether the term ended it, and the value that each logged quantity had at its end. Withcompute_final_obs,extras["final_obs"]holds the observations that the worlds that ended had before their reset, also as views; only the rows of the worlds that ended are of this step.Methods:
- static __new__(cls, *args: Any, **kwargs: Any) Any#
- class isaaclab_experimental.envs.MDP2RLEnvCfg[source]#
Bases:
MDP2EnvCfgConfiguration for
MDP2RLEnv.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(sim: ~isaaclab.sim.simulation_cfg.SimulationCfg = <factory>, scene: ~isaaclab.scene.interactive_scene_cfg.InteractiveSceneCfg = <factory>, task: ~isaaclab_experimental.mdp2.config.TaskCfg | list[~isaaclab_experimental.mdp2.config.TaskCfg] = <factory>, task_worlds: list[int] | None = <factory>, decimation: int = <factory>, backend: ~isaaclab_experimental.mdp2.compiled.Backend = <factory>, seed: int | None = <factory>, capture: bool = <factory>, step: list[~isaaclab_experimental.envs.mdp2_env_cfg.Step | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, reset_step: list[~isaaclab_experimental.envs.mdp2_env_cfg.ResetStep | ~isaaclab_experimental.envs.mdp2_env_cfg.Stage] = <factory>, verify_capture: int = <factory>, is_finite_horizon: bool = <factory>, compute_final_obs: bool = <factory>, log_dir: str | None = <factory>, video_recorders: list = <factory>) None#
- class isaaclab_experimental.envs.ResetStep[source]#
Built-in work of a reset, in
MDP2EnvCfg.reset_step.
- class isaaclab_experimental.envs.Stage[source]#
User work in a control step or a reset, at its place in
MDP2EnvCfg.steporMDP2EnvCfg.reset_step, such as kernels that update a scene field the MDP reads.fn(env)may read and write anything, through the environment, its scene, and its MDP state. A capturable stage runs inside the step’s CUDA graph: it is called once, at capture, and its launches replay every step, so it must not read values on the host or draw from Torch’s random generators, and Python branches take the path they took at capture. Check it withMDP2EnvCfg.verify_capture. A stage that is not capturable runs on the host between the graphs. After each stage, the simulation applies the model and state changes it made through the assets.Methods:
- classmethod __new__(*args, **kwargs)#