isaaclab_experimental.envs

isaaclab_experimental.envs#

Sub-package for environment definitions.

Environments define the interface between the agent and the simulation. In the simplest case, the environment provides the agent with the current observations and executes the actions provided by the agent. However, the environment can also provide additional information such as the current reward, done flag, and information about the current episode.

There are two types of environment designing workflows:

  • Manager-based: The environment is decomposed into individual components (or managers) for different aspects (such as computing observations, applying actions, and applying randomization. The users mainly configure the managers and the environment coordinates the managers and calls their functions.

  • Direct: The user implements all the necessary functionality directly into a single class directly without the need for additional managers.

Based on these workflows, there are the following environment classes for single and multi-agent RL:

Single-Agent RL:

  • ManagerBasedEnv: The manager-based workflow base environment which only provides the agent with the current observations and executes the actions provided by the agent.

  • ManagerBasedRLEnv: The manager-based workflow RL task environment which besides the functionality of the base environment also provides additional Markov Decision Process (MDP) related information such as the current reward, done flag, and information.

  • DirectRLEnv: The direct workflow RL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

Multi-Agent RL (MARL):

  • DirectMARLEnv: The direct workflow MARL task environment which provides implementations for implementing scene setup, computing dones, performing resets, and computing reward and observation.

For more information about the workflow design patterns, see the Task Design Workflows section.