In-Process Policies#

An in-process policy computes its actions inside the simulation process. It needs no server and no network connection, so it is the fastest way to check that an environment loads, to replay recorded data, or to run a small trained network.

Arena includes three in-process policies. zero_action returns zero actions and needs no model weights, so the robot stays still while the environment loads and renders. replay plays back one episode from an HDF5 recording. It reports the episode length, so the rollout ends when the recording does. rsl_rl runs a trained RSL-RL checkpoint and loads the params/agent.yaml saved with it.

All three live in isaaclab_arena/policy/. ZeroActionPolicy is the smallest complete example of the policy contract:

zero_action_policy.py
import gymnasium as gym
import torch
from dataclasses import dataclass
from gymnasium.spaces.dict import Dict as GymSpacesDict

from isaaclab_arena.assets.register import register_policy
from isaaclab_arena.policy.policy_base import PolicyBase, PolicyCfg


@dataclass
class ZeroActionPolicyCfg(PolicyCfg):
    """Configure a policy that always returns zero actions."""


@register_policy
class ZeroActionPolicy(PolicyBase[ZeroActionPolicyCfg]):

    name = "zero_action"

    def __init__(self, config: ZeroActionPolicyCfg):
        """
        Initialize ZeroActionPolicy.

        Args:
            config: Typed policy configuration.
        """
        super().__init__(config)

    def get_action(self, env: gym.Env, observation: GymSpacesDict) -> torch.Tensor:
        """
        Always returns a zero action.
        """
        return torch.zeros(env.action_space.shape, device=torch.device(env.unwrapped.device))

Run an in-process policy#

Select the policy with --policy_type. Each field of its configuration becomes a flag:

python isaaclab_arena/evaluation/policy_runner.py \
  --policy_type replay \
  --replay_file_path demo.hdf5 \
  <environment> <environment arguments>

Exploring Environment Variations runs zero_action through the Experiment Runner instead.

Add your own#

Write an in-process policy when inference is cheap and its dependencies fit in the Arena image. Examples are a scripted controller, an imitation-learning checkpoint, or a policy from another RL library. Follow Write your own policy. If the policy keeps state across steps, such as a recurrent hidden state or a queued action chunk, clear it for the affected environments in reset(env_ids). If it plays back data of a fixed length, implement has_length() and length().

When a model brings its own dependencies or needs its own GPU or machine, keep it out of the simulation process and run it as a server instead.

Next steps#

Continue to Server–Client Policies to evaluate a foundation model running in a separate process.