Wrapping environments

A gymnasium.Wrapper changes an environment’s observations, rewards, or execution without changing the environment implementation. Common uses include recording videos and enforcing time limits.

Both ManagerBasedRLEnv and DirectRLEnv implement gymnasium.Env. Create the environment first, then apply each wrapper with env = wrapper(env, *args, **kwargs). The order matters: each wrapper receives the output of the one inside it.

For example, here is how you would wrap an environment to enforce that reset is called before step or render:

import gymnasium as gym

from isaaclab.app import launch_simulation

import isaaclab_tasks  # noqa: F401
from isaaclab_tasks.utils import load_cfg_from_registry

cfg = load_cfg_from_registry("Isaac-Reach-Franka", "env_cfg_entry_point")
# start the simulation runtime this configuration needs, and stop it on exit
with launch_simulation(cfg):
    # create base environment
    env = gym.make("Isaac-Reach-Franka", cfg=cfg)
    # wrap environment to enforce that reset is called before step
    env = gym.wrappers.OrderEnforcing(env)

Wrapper for recording videos

The gymnasium.wrappers.RecordVideo wrapper can be used to record videos of the environment. The wrapper takes a video_dir argument, which specifies where to save the videos. The videos are saved in mp4 format at specified intervals for specified number of environment steps or episodes.

To use the wrapper, you need to first install ffmpeg. On Ubuntu, you can install it by running:

sudo apt-get install ffmpeg

Attention

By default, when running an environment in headless mode, the Omniverse viewport is disabled. This is done to improve performance by avoiding unnecessary rendering.

We notice the following performance in different rendering modes with the Isaac-Reach-Franka environment using an RTX 3090 GPU:

• No GUI execution without off-screen rendering enabled: ~65,000 FPS

• No GUI execution with off-screen enabled: ~57,000 FPS

• GUI execution with full rendering: ~13,000 FPS

The viewport camera used for rendering is the default camera in the scene called "/OmniverseKit_Persp". The camera’s pose and image resolution can be configured through the ViewerCfg class.

Default parameters of the ViewerCfg class:

@configclass
class ViewerCfg:
    """Configuration of the scene viewport camera.

    .. deprecated::
        :class:`ViewerCfg` is deprecated and will be removed in a future release.
        Configure the viewport camera via :class:`~isaaclab.visualizers.VisualizerCfg`
        on :attr:`~isaaclab.sim.SimulationCfg.visualizer_cfgs` instead. Select its Kit backend:

    .. code-block:: python

        from isaaclab.sim import SimulationCfg
        from isaaclab.visualizers import VisualizerCfg
        from isaaclab_visualizers.kit import KitBackendCfg

        sim_cfg = SimulationCfg(
            visualizer_cfgs=VisualizerCfg(eye=(7.5, 7.5, 7.5), lookat=(0.0, 0.0, 0.0), backend=KitBackendCfg())
        )
    """

    eye: tuple[float, float, float] = (7.5, 7.5, 7.5)
    """Initial camera position (in m). Default is (7.5, 7.5, 7.5)."""

    lookat: tuple[float, float, float] = (0.0, 0.0, 0.0)
    """Initial camera target position (in m). Default is (0.0, 0.0, 0.0)."""

    cam_prim_path: str = "/OmniverseKit_Persp"
    """The camera prim path to record images from. Default is "/OmniverseKit_Persp"."""

    resolution: tuple[int, int] = (1280, 720)
    """The resolution (width, height) of the camera. Default is (1280, 720)."""

    origin_type: Literal["world", "env", "asset_root", "asset_body"] = "world"
    """The frame in which the camera position (eye) and target (lookat) are defined. Default is "world"."""

    env_index: int = 0
    """The environment index for frame origin. Default is 0."""

    asset_name: str | None = None
    """The asset name in the interactive scene for the frame origin. Default is None."""

    body_name: str | None = None
    """The name of the body in :attr:`asset_name` for the frame origin. Default is None."""

    def __post_init__(self) -> None:
        # Bare ViewerCfg() declarations stay silent; warn once for any customized field.
        for field in fields(self):
            default = field.default if field.default_factory is MISSING else field.default_factory()
            if field.init and default is not MISSING and vars(self)[field.name] != default:
                warnings.warn(
                    "ViewerCfg is deprecated and will be removed in a future release. "
                    "Use VisualizerCfg on SimulationCfg.visualizer_cfgs instead.",
                    DeprecationWarning,
                    stacklevel=2,
                )
                break

To record videos, add a VideoRecorderCfg to the environment configuration. It records from a visualizer (or a camera sensor) into mp4 clips while the environment steps, so no wrapper or render_mode is needed, and the runtime the visualizer needs starts automatically.

As an example, the following code records 200-step clips of the Isaac-Reach-Franka environment from a Kit visualizer every 1500 steps into the videos/train folder. See Recording video for the other sources and clip options.

import gymnasium as gym

from isaaclab_visualizers.kit import KitBackendCfg

from isaaclab.app import launch_simulation
from isaaclab.envs.utils.video_recorder_cfg import VideoRecorderCfg

# record from a Kit visualizer with this camera pose
env_cfg.sim.visualizer_cfgs = KitBackendCfg(eye=(1.0, 1.0, 1.0), lookat=(0.0, 0.0, 0.0))
env_cfg.video_recorders = [
    VideoRecorderCfg(source="viz:kit", output_dir="videos/train", video_length=200, video_interval=1500)
]
# select the Kit visualizer, as ``--viz kit`` does; the config above only sets its camera pose
with launch_simulation(env_cfg, {"visualizer": "kit"}):
    env = gym.make(task_name, cfg=env_cfg)

Wrapper for learning frameworks

Every learning framework has its own API for interacting with environments. For example, the Stable-Baselines3 library uses the gym.Env interface to interact with environments. However, libraries like RL-Games, RSL-RL or SKRL use their own API for interfacing with a learning environments. Since there is no one-size-fits-all solution, we do not base the ManagerBasedRLEnv and DirectRLEnv classes on any particular learning framework’s environment definition. Instead, we implement wrappers to make it compatible with the learning framework’s environment definition.

As an example of how to use the RL task environment with Stable-Baselines3:

from isaaclab_rl.sb3 import Sb3VecEnvWrapper

# create isaac-env instance
env = gym.make(task_name, cfg=env_cfg)
# wrap around environment for stable baselines
env = Sb3VecEnvWrapper(env)

Stable-Baselines3 requires finite continuous action bounds. When the environment has an unbounded action space, Sb3VecEnvWrapper exposes normalized [-1, 1] bounds to Stable-Baselines3 without changing the underlying environment. Set action_bounds when the policy uses a different finite action domain.

Caution

Wrapping the environment with the respective learning framework’s wrapper should happen in the end, i.e. after all other wrappers have been applied. This is because the learning framework’s wrapper modifies the interpretation of environment’s APIs which may no longer be compatible with gymnasium.Env.

Adding new wrappers

All new wrappers should be added to the isaaclab_rl module. They should check that the underlying environment is an instance of isaaclab.envs.ManagerBasedRLEnv or DirectRLEnv before applying the wrapper. This can be done by using the unwrapped() property.

We include a set of wrappers in this module that can be used as a reference to implement your own wrappers. If you implement a new wrapper, please consider contributing it to the framework by opening a pull request.