isaaclab_experimental.mdp2

Contents

isaaclab_experimental.mdp2#

MDP2: task configs compiled into an immutable MDP, run by an environment through functional steps.

See MDP2: Compiled MDPs for a guide.

Terms and task configuration#

Term

A reusable calculation with separate state per use.

Implementation

Backend hooks of a term.

State

A float32 buffer that a term keeps across steps, with one value per world and element by default.

TermContext

Data that an init hook reads and writes, as Torch views over its worlds.

Entity

A scene entity.

FieldSelector

Selects a field of a scene entity (robot.joint_pos) or of the MDP (mdp2.commands.target).

SubsetSelector

Selects the elements of an entity dimension whose names match regular expressions, in entity order.

TaskCfg

Configuration of the MDP of a task.

DirectAction

Configuration for an action that writes scale * action + offset to an entity field.

Observation

Configuration for an observation term, whose output is flattened per world into its group's buffer.

Reward

Configuration for a reward term, whose output adds weight * value * step_dt to the reward.

Termination

Configuration for a termination term, which ends the episode of every world where its output is true.

TimeLimit

Truncates episodes after episode_length_s seconds.

Command

Configuration for a command term, which is resampled on reset and whenever its resampling timer expires.

Event

Configuration for an event term, which writes its outputs for the worlds selected by mask.

EventMode

When an event runs: once at startup, on reset, or at random intervals.

Quantity

Configuration for a quantity: a value per world that other terms read as mdp2.quantities.<name>.

UniformNoise

Noise drawn uniformly from [n_min, n_max] for each element, like MDP1's UniformNoiseCfg.

term_state

Select the state name of another term use, as in term_state("rewards.success", "succeeded").

Building and running#

MDPBuilder

Collects term uses, lets users edit their order, and compiles them with finalize().

MDP

An immutable, compiled MDP, created with MDPBuilder.finalize().

MDPState

Data of an MDP that changes as it runs, such as episode lengths, term state, timers, and random streams.

Transition

Results of one step, written into buffers that the caller owns.

Backend

Backend that runs the term implementations.

CapturedMDP

The runtime functions of an MDP, bound to fixed buffers and replayed as CUDA graphs.

startup

Run the init hooks of the terms, then the startup events, once, in every world.

apply_actions

Store the actions, shape (num_worlds, action_dim), and write them to their target fields.

evaluate

Compute terminations and rewards, resample due commands, run due interval events, and observe.

reset

Reset the masked worlds: restart timers, run reset events, resample commands, then reset term state.

observe

Compute observations for the masked worlds and leave the others unchanged.

Scene data#

SceneData

Interface through which an MDP reads and writes scene data.

SceneSchema

Description of what a scene provides to an MDP.

EntitySchema

Element names and field layouts of one scene entity.

FieldSchema

Layout of an entity field, an array of shape (num_worlds, *dims).

class isaaclab_experimental.mdp2.Term[source]#

A reusable calculation with separate state per use. A function is short for Implementation(compute=fn).

Methods:

__new__(*args, **kwargs)

__init__([torch, warp, shape, state, ...])

classmethod __new__(*args, **kwargs)#
__init__(torch: ~isaaclab_experimental.mdp2.terms.Implementation | ~collections.abc.Callable | None = None, warp: ~isaaclab_experimental.mdp2.terms.Implementation | ~collections.abc.Callable | None = None, shape: tuple[str | int, ...] = ('worlds', ), state: dict[str, ~isaaclab_experimental.mdp2.terms.State] = <factory>, random: bool = False, init: ~collections.abc.Callable | None = None, capturable: bool = True) → None#
class isaaclab_experimental.mdp2.Implementation[source]#

Backend hooks of a term. Hooks take any of the term’s inputs, state, params, and rng by name.

Fields arrive as arrays of shape (num_worlds, ...), and subsets as int32 index arrays.

Torch hooks are batched over worlds:

  • compute(**args) returns the output. Quantities, terminations, and rewards are computed once per step, so their compute may also advance the term’s state in place. An observation’s must not: observations are computed again after a reset, and continuing worlds must get the same sample. Nor may the compute of a quantity that observations read, unless repeating it leaves the state as it is.

  • update(**args) advances state in place, once per step after compute, for terms whose compute must not. A hook that writes a field that other world configs share, such as a command, also takes mask, the worlds where its op runs (None for every world).

  • reset(mask, **args) resets the masked worlds’ state, after the reset events ran and the commands were resampled. Without it, state is set to initial.

Warp hooks are wp.func s that handle one world, called from the kernels generated for the MDP:

  • compute(world, *indices, **args) returns one output element, with one index per output dimension after "worlds"; bool for terminations and float32 otherwise.

  • update(world, **args) and reset(world, **args) act on one world; reset runs for reset worlds only.

  • A func gets rng as its world’s int32 seed. It must only write its own world, and may read other worlds only where no term writes.

A Python hook, a Torch hook or a Warp hook that launches kernels itself, may also take scene entities bound as inputs, such as inputs={"robot": robot}, to call any of their methods, and cache, a dict that the use keeps across calls for host objects. A capturable Python hook runs once, at capture, and its launches replay: it must not read values on the host or draw from Torch’s random generators, and its Python branches take the path they took at capture. Mark the term capturable=False otherwise: it then runs on the host between the graphs.

A Warp hook can instead be a function that launches kernels itself, with the Torch arguments plus out for compute, an array of the output shape to fill for every world, and rng as the stream of every world. It runs between the generated kernels.

Event and command terms are compute(mask, **args), or a func compute(world, **args) run for the selected worlds, and write their outputs in place. A command gets its value as command: compute resamples it and writes the command’s state, update may change it every step, and commands have no reset hook.

Methods:

__new__(*args, **kwargs)

__init__(compute[, update, reset])

classmethod __new__(*args, **kwargs)#
__init__(compute: Callable, update: Callable | None = None, reset: Callable | None = None) → None#
class isaaclab_experimental.mdp2.State[source]#

A float32 buffer that a term keeps across steps, with one value per world and element by default.

Methods:

__new__(*args, **kwargs)

__init__([shape, initial, persistent])

classmethod __new__(*args, **kwargs)#
__init__(shape: tuple[str | int, ...] = ('worlds',), initial: float = 0.0, persistent: bool = False) → None#
class isaaclab_experimental.mdp2.TermContext[source]#

Data that an init hook reads and writes, as Torch views over its worlds.

Methods:

entity(argument)

Return the scene object, such as an articulation, of an entity input or of the entity an input selects.

__init__(num_worlds, step_dt, inputs, ...)

entity(argument: str) → Any[source]#

Return the scene object, such as an articulation, of an entity input or of the entity an input selects.

__init__(num_worlds: int, step_dt: float, inputs: dict[str, torch.Tensor], outputs: dict[str, torch.Tensor], state: dict[str, torch.Tensor], rng: torch.Tensor | None, selectors: Mapping[str, FieldSelector | SubsetSelector | Entity], scene_data: SceneData) → None#
class isaaclab_experimental.mdp2.Entity[source]#

A scene entity. Accessing an attribute selects one of its fields, as in robot.joint_vel.

Methods:

joints(*patterns)

Select the joints whose names match any of the patterns.

bodies(*patterns)

Select the bodies whose names match any of the patterns.

__init__(name[, scene])

joints(*patterns: str) → SubsetSelector[source]#

Select the joints whose names match any of the patterns.

bodies(*patterns: str) → SubsetSelector[source]#

Select the bodies whose names match any of the patterns.

__init__(name: str, scene: bool = True) → None#
class isaaclab_experimental.mdp2.FieldSelector[source]#

Selects a field of a scene entity (robot.joint_pos) or of the MDP (mdp2.commands.target).

Methods:

__new__(*args, **kwargs)

__init__(entity, field[, scene])

classmethod __new__(*args, **kwargs)#
__init__(entity: str, field: str, scene: bool = True) → None#
class isaaclab_experimental.mdp2.SubsetSelector[source]#

Selects the elements of an entity dimension whose names match regular expressions, in entity order.

Bound to an input, as in inputs={"joint_ids": robot.joints(".*_hip")}, it passes the term an int32 index array. The argument name also becomes a dimension that the term’s shapes can use.

Methods:

__new__(*args, **kwargs)

__init__(entity, dim, patterns)

classmethod __new__(*args, **kwargs)#
__init__(entity: str, dim: str, patterns: tuple[str, ...]) → None#
class isaaclab_experimental.mdp2.TaskCfg[source]#

Configuration of the MDP of a task. Setting a term use to None disables it.

Methods:

__new__(*args, **kwargs)

__init__([actions, commands, quantities, ...])

classmethod __new__(*args, **kwargs)#
__init__(actions: dict[str, ~isaaclab_experimental.mdp2.config.DirectAction | None] = <factory>, commands: dict[str, ~isaaclab_experimental.mdp2.config.Command | None] = <factory>, quantities: dict[str, ~isaaclab_experimental.mdp2.config.Quantity | None] = <factory>, observations: dict[str, dict[str, ~isaaclab_experimental.mdp2.config.Observation | None]] = <factory>, rewards: dict[str, ~isaaclab_experimental.mdp2.config.Reward | None] = <factory>, terminations: dict[str, ~isaaclab_experimental.mdp2.config.Termination | ~isaaclab_experimental.mdp2.config.TimeLimit | None] = <factory>, events: dict[str, ~isaaclab_experimental.mdp2.config.Event | None] = <factory>) → None#
class isaaclab_experimental.mdp2.DirectAction[source]#

Configuration for an action that writes scale * action + offset to an entity field.

Methods:

__new__(*args, **kwargs)

__init__(target[, scale, offset, subset])

classmethod __new__(*args, **kwargs)#
__init__(target: FieldSelector, scale: float = 1.0, offset: float | FieldSelector = 0.0, subset: SubsetSelector | None = None) → None#
class isaaclab_experimental.mdp2.Observation[source]#

Configuration for an observation term, whose output is flattened per world into its group’s buffer.

As in MDP1, the noise is added first, then the value is clipped, then scaled. With a history, the group holds the last history_length values, oldest first, and a new episode starts with its first value repeated.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, noise, ...])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, noise: ~isaaclab_experimental.mdp2.config.UniformNoise | None = None, clip: tuple[float, float] | None = None, scale: float | None = None, history_length: int = 0) → None#
class isaaclab_experimental.mdp2.Reward[source]#

Configuration for a reward term, whose output adds weight * value * step_dt to the reward.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, weight])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, weight: float = 1.0) → None#
class isaaclab_experimental.mdp2.Termination[source]#

Configuration for a termination term, which ends the episode of every world where its output is true.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, time_out])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, time_out: bool = False) → None#
class isaaclab_experimental.mdp2.TimeLimit[source]#

Truncates episodes after episode_length_s seconds.

Methods:

__new__(*args, **kwargs)

__init__(episode_length_s)

classmethod __new__(*args, **kwargs)#
__init__(episode_length_s: float) → None#
class isaaclab_experimental.mdp2.Command[source]#

Configuration for a command term, which is resampled on reset and whenever its resampling timer expires.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, ...])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, resampling_time_range: tuple[float, float] | None = None) → None#
class isaaclab_experimental.mdp2.Event[source]#

Configuration for an event term, which writes its outputs for the worlds selected by mask.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, outputs, ...])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, outputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector] = <factory>, mode: ~isaaclab_experimental.mdp2.config.EventMode = EventMode.RESET, interval_range_s: tuple[float, float] | None = None, commit: bool | None = None) → None#
class isaaclab_experimental.mdp2.EventMode[source]#

When an event runs: once at startup, on reset, or at random intervals.

class isaaclab_experimental.mdp2.Quantity[source]#

Configuration for a quantity: a value per world that other terms read as mdp2.quantities.<name>.

Quantities are computed once per step, before the terminations and rewards, so that terms share a calculation instead of repeating it. A quantity keeps its value until the next step: reset events read what the episode that ended left, such as whether it succeeded. A quantity that observations or action offsets read is the exception: it is computed again after a reset, with the quantities that it reads.

Methods:

__new__(*args, **kwargs)

__init__(term[, inputs, params, log])

classmethod __new__(*args, **kwargs)#
__init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, log: bool = False) → None#
class isaaclab_experimental.mdp2.UniformNoise[source]#

Noise drawn uniformly from [n_min, n_max] for each element, like MDP1’s UniformNoiseCfg.

Methods:

__new__(*args, **kwargs)

__init__(n_min, n_max[, scale])

classmethod __new__(*args, **kwargs)#
__init__(n_min: float, n_max: float, scale: FieldSelector | None = None) → None#
isaaclab_experimental.mdp2.term_state(use: str, name: str) → FieldSelector[source]#

Select the state name of another term use, as in term_state("rewards.success", "succeeded").

Terms read it as an input; only the use that owns it writes it. Reset events see the state of the episode that ended, since term state resets after them. World configs that define the use differently share its state, each in its own worlds, so its shapes must agree; state that the worlds share, such as a table filled by init, is filled by each of them in turn.

class isaaclab_experimental.mdp2.MDPBuilder[source]#

Collects term uses, lets users edit their order, and compiles them with finalize().

Uses run phase by phase. Within a phase they run in the order they were added, unless the order is edited with move_before(), move_after(), or set_order(). Every edit is checked so that a use that reads a field still runs after the uses of its phase that write it. Action and observation columns follow the schedule order.

Methods:

__init__(schema)

replicate(builder, num_worlds)

Run the task of another builder in the next num_worlds worlds.

add_task(task)

Add all term uses of a task in declaration order, skipping uses set to None.

entry(*path)

Return the entry at a path, for example builder.entry("rewards", "alive").

schedule()

Return all entries in execution order: by phase, then by their order within the phase.

move_before(entry, anchor)

Run entry just before anchor, which must be in the same phase.

move_after(entry, anchor)

Run entry just after anchor, which must be in the same phase.

set_order(phase, entries)

Set the execution order of all entries of a phase.

describe()

Return the schedule as readable text, listing each use with the fields it reads and writes.

inspect(entry[, backend])

Compile the schedule and return the op of an entry, or None for a time limit.

preview_segments([backend])

Return how the current order splits each runtime function into CUDA graphs and eager steps.

finalize([device, backend, diagnostics, ...])

Validate the schedule and compile it into an immutable MDP.

__init__(schema: SceneSchema) → None[source]#
replicate(builder: MDPBuilder, num_worlds: int) → None[source]#

Run the task of another builder in the next num_worlds worlds.

The builder’s schedule is copied, so later edits to it do not apply. Once worlds are assigned, every world must be, and term uses are added to the replicated builders rather than to this one.

add_task(task: TaskCfg) → None[source]#

Add all term uses of a task in declaration order, skipping uses set to None.

entry(*path: str) → ScheduleEntry[source]#

Return the entry at a path, for example builder.entry("rewards", "alive").

schedule() → tuple[ScheduleEntry, ...][source]#

Return all entries in execution order: by phase, then by their order within the phase.

move_before(entry: ScheduleEntry, anchor: ScheduleEntry) → None[source]#

Run entry just before anchor, which must be in the same phase.

move_after(entry: ScheduleEntry, anchor: ScheduleEntry) → None[source]#

Run entry just after anchor, which must be in the same phase.

set_order(phase: Phase, entries: list[ScheduleEntry]) → None[source]#

Set the execution order of all entries of a phase.

describe() → str[source]#

Return the schedule as readable text, listing each use with the fields it reads and writes.

inspect(entry: ScheduleEntry, backend: Backend | str = Backend.TORCH) → ActionOp | TermOp | None[source]#

Compile the schedule and return the op of an entry, or None for a time limit.

preview_segments(backend: Backend | str = Backend.TORCH) → dict[str, tuple[Segment, ...]][source]#

Return how the current order splits each runtime function into CUDA graphs and eager steps.

finalize(device: str = 'cpu', backend: Backend | str = Backend.TORCH, diagnostics: bool = False, final_observations: bool = False) → MDP[source]#

Validate the schedule and compile it into an immutable MDP.

Raises a single ValueError that lists every error found. Editing the builder afterwards does not change the returned MDP. With diagnostics, evaluate() also writes the reward and termination flag of each term. With final_observations, it also writes the observations of the worlds that ended, before their reset, to Transition.final_observations, in the kernels that compute them.

class isaaclab_experimental.mdp2.MDP[source]#

An immutable, compiled MDP, created with MDPBuilder.finalize().

Attributes:

narrow_action_dims

Action counts of the world configs with fewer actions than the widest.

term_ops

Ops of every phase except actions, in schedule order.

schedule

Names of all ops in schedule order.

segments

runs of steps that are captured as one CUDA graph, or run eagerly.

is_fully_capturable

Whether every runtime function can be captured as a single CUDA graph.

startup_writes

Scene fields that the startup events write, which the scene commits after startup.

reset_writes

Scene fields that the reset events write, which the scene commits after a reset.

interval_writes

Scene fields that the interval events write, which the scene commits in the worlds where one ran.

Methods:

describe()

Return the compiled MDP as readable text: ops by phase with their layout, then each function's steps.

state(scene_data[, seed])

Create the runtime state, bound to the data views of a scene with the same schema.

observations()

Allocate one zeroed observation buffer per group, of shape (num_worlds, group_dim).

transition()

Allocate the zeroed buffers of a transition, with the optional buffers that the MDP writes.

__init__(schema, device, backend, ops, plan, ...)

property narrow_action_dims: tuple[int, ...][source]#

Action counts of the world configs with fewer actions than the widest.

property term_ops: tuple[TermOp, ...][source]#

Ops of every phase except actions, in schedule order.

property schedule: tuple[str, ...][source]#

Names of all ops in schedule order.

property segments: dict[str, tuple[Segment, ...]][source]#

runs of steps that are captured as one CUDA graph, or run eagerly.

The Warp backend may need fewer graphs: it runs a hook early when nothing between depends on it.

Type:

Segments of each runtime function

property is_fully_capturable: bool[source]#

Whether every runtime function can be captured as a single CUDA graph. Startup runs once, uncaptured.

property startup_writes: frozenset[FieldSelector][source]#

Scene fields that the startup events write, which the scene commits after startup.

property reset_writes: frozenset[FieldSelector][source]#

Scene fields that the reset events write, which the scene commits after a reset.

property interval_writes: frozenset[FieldSelector][source]#

Scene fields that the interval events write, which the scene commits in the worlds where one ran.

describe() → str[source]#

Return the compiled MDP as readable text: ops by phase with their layout, then each function’s steps.

state(scene_data: SceneData, seed: int = 0) → MDPState[source]#

Create the runtime state, bound to the data views of a scene with the same schema.

observations() → dict[str, ProxyArray][source]#

Allocate one zeroed observation buffer per group, of shape (num_worlds, group_dim).

transition() → Transition[source]#

Allocate the zeroed buffers of a transition, with the optional buffers that the MDP writes.

__init__(schema: SceneSchema, device: str, backend: Backend, ops: Mapping[Phase, tuple[ActionOp | TermOp, ...]], plan: ExecutionPlan, action_dim: int, observation_dims: Mapping[str, int], max_episode_length: int | None, streams: tuple[str, ...], subsets: Mapping[SubsetSelector, tuple[int, ...]], diagnostics: bool, final_observations: bool, world_configs: tuple[WorldConfig, ...], launches: Launches | None = None) → None#
class isaaclab_experimental.mdp2.MDPState[source]#

Data of an MDP that changes as it runs, such as episode lengths, term state, timers, and random streams.

Every array is in arrays, under the key that the backends use for it:

  • a field or subset selector: the selected fields of the scene, the fields of the MDP (mdp2.last_action, mdp2.commands.<name>, …), and the state of every term use, mdp2.term_state(use, name);

  • (mdp2.last_action, action_dim): the first columns of the last actions, which the terms of a world config with action_dim actions read if other world configs have more;

  • "rng": the random streams, (num_streams, num_worlds), see rng;

  • "reward_weights": (num_worlds, num_rewards), which may change while the MDP runs, also in a CUDA graph;

  • ("timer", op): seconds left per world, for a timed op;

  • ("history", op): (num_worlds, history_length, size), for an observation with a history;

  • ("scratch", op): the output of a Python hook in a Warp MDP;

  • ("selected", op): the worlds that the hook of a timed op, or of an op active in some worlds, acts on;

  • ("active", op): the worlds where an op is active, for an op that is active in only some;

  • "interval_worlds": the worlds where an interval event that writes scene fields ran in the last evaluate, for an MDP with such events.

Attributes:

episode_length

Control steps since the last reset, (num_worlds,).

actions

The last actions, (num_worlds, action_dim).

Methods:

__init__(arrays, arguments, scene_data[, caches])

property episode_length: ProxyArray[source]#

Control steps since the last reset, (num_worlds,).

property actions: ProxyArray[source]#

The last actions, (num_worlds, action_dim).

__init__(arrays: dict[~collections.abc.Hashable, ~isaaclab.utils.warp.proxy_array.ProxyArray], arguments: dict[str, dict[str, ~typing.Any]], scene_data: ~isaaclab_experimental.mdp2.scene.SceneData, caches: dict[str, dict] = <factory>) → None#
class isaaclab_experimental.mdp2.Transition[source]#

Results of one step, written into buffers that the caller owns.

Methods:

__new__(*args, **kwargs)

__init__(rewards, terminated, truncated, ...)

classmethod __new__(*args, **kwargs)#
__init__(rewards: ProxyArray, terminated: ProxyArray, truncated: ProxyArray, done: ProxyArray, observations: dict[str, ProxyArray], reward_terms: ProxyArray | None = None, termination_terms: ProxyArray | None = None, final_observations: dict[str, ProxyArray] | None = None) → None#
class isaaclab_experimental.mdp2.Backend[source]#

Backend that runs the term implementations.

class isaaclab_experimental.mdp2.CapturedMDP[source]#

The runtime functions of an MDP, bound to fixed buffers and replayed as CUDA graphs.

Consecutive capturable launches of a function are captured as one graph, and hooks that cannot be captured run eagerly between the graphs. Capturing runs nothing. Before each call, write the actions and the mask of the worlds to reset or observe into the bound actions and mask buffers. On CPU, the launches run eagerly.

Methods:

__init__(mdp, state, transition, actions, mask)

apply_actions()

Store the bound actions and write them to their target fields.

evaluate()

Compute terminations, rewards, and successor observations into the bound transition.

reset()

Reset the worlds selected by the bound mask.

observe()

Compute the observations of the worlds selected by the bound mask into the bound transition.

__init__(mdp: MDP, state: MDPState, transition: Transition, actions: ProxyArray, mask: ProxyArray, commit: Callable[[], None] | None = None) → None[source]#
apply_actions() → None[source]#

Store the bound actions and write them to their target fields.

evaluate() → None[source]#

Compute terminations, rewards, and successor observations into the bound transition.

reset() → None[source]#

Reset the worlds selected by the bound mask.

observe() → None[source]#

Compute the observations of the worlds selected by the bound mask into the bound transition.

isaaclab_experimental.mdp2.startup(mdp: MDP, state: MDPState) → None[source]#

Run the init hooks of the terms, then the startup events, once, in every world.

isaaclab_experimental.mdp2.apply_actions(mdp: MDP, state: MDPState, actions: ProxyArray) → None[source]#

Store the actions, shape (num_worlds, action_dim), and write them to their target fields.

isaaclab_experimental.mdp2.evaluate(mdp: MDP, state: MDPState, transition: Transition) → None[source]#

Compute terminations and rewards, resample due commands, run due interval events, and observe.

isaaclab_experimental.mdp2.reset(mdp: MDP, state: MDPState, mask: ProxyArray, commit: Callable[[], None] | None = None) → None[source]#

Reset the masked worlds: restart timers, run reset events, resample commands, then reset term state.

commit makes the scene see the writes of the reset events so far. It is called before every event that asks for it, and tasks without such events need none. The caller commits the writes once more after the reset.

isaaclab_experimental.mdp2.observe(mdp: MDP, state: MDPState, observations: dict[str, ProxyArray], mask: ProxyArray) → None[source]#

Compute observations for the masked worlds and leave the others unchanged.

Recomputes the quantities that observations read, and fills the observation histories of new episodes, in the masked worlds. Changes no other MDP state.

class isaaclab_experimental.mdp2.SceneData[source]#

Interface through which an MDP reads and writes scene data. The environment owns the scene.

Methods:

data(entity, field)

Return a live view of an entity field, of shape (num_worlds, ...).

entity(name)

Return the scene object of an entity, such as an articulation, for Python hooks that take it.

commit(mask, fields)

Make the simulation see the fields that events wrote in the masked worlds, and refresh derived data.

__init__(*args, **kwargs)

data(entity: str, field: str) → ProxyArray[source]#

Return a live view of an entity field, of shape (num_worlds, ...).

entity(name: str) → Any[source]#

Return the scene object of an entity, such as an articulation, for Python hooks that take it.

commit(mask: array, fields: Collection[FieldSelector]) → None[source]#

Make the simulation see the fields that events wrote in the masked worlds, and refresh derived data.

__init__(*args, **kwargs)#
class isaaclab_experimental.mdp2.SceneSchema[source]#

Description of what a scene provides to an MDP. The builder resolves selectors against it.

Methods:

__new__(*args, **kwargs)

__init__(num_worlds, step_dt, entities)

classmethod __new__(*args, **kwargs)#
__init__(num_worlds: int, step_dt: float, entities: Mapping[str, EntitySchema]) → None#
class isaaclab_experimental.mdp2.EntitySchema[source]#

Element names and field layouts of one scene entity.

Methods:

__new__(*args, **kwargs)

__init__(names, fields)

classmethod __new__(*args, **kwargs)#
__init__(names: Mapping[str, tuple[str, ...]], fields: Mapping[str, FieldSchema]) → None#
class isaaclab_experimental.mdp2.FieldSchema[source]#

Layout of an entity field, an array of shape (num_worlds, *dims).

Methods:

__new__(*args, **kwargs)

__init__([dims, dtype, writable, derived_from])

classmethod __new__(*args, **kwargs)#
__init__(dims: tuple[str, ...] = (), dtype: type = <class 'warp._src.types.float32'>, writable: bool = False, derived_from: tuple[str, ...] = ()) → None#