isaaclab_experimental.mdp2#
MDP2: task configs compiled into an immutable MDP, run by an environment through functional steps.
See MDP2: Compiled MDPs for a guide.
Terms and task configuration#
A reusable calculation with separate state per use. |
|
Backend hooks of a term. |
|
A |
|
Data that an init hook reads and writes, as Torch views over its worlds. |
|
A scene entity. |
|
Selects a field of a scene entity ( |
|
Selects the elements of an entity dimension whose names match regular expressions, in entity order. |
|
Configuration of the MDP of a task. |
|
Configuration for an action that writes |
|
Configuration for an observation term, whose output is flattened per world into its group's buffer. |
|
Configuration for a reward term, whose output adds |
|
Configuration for a termination term, which ends the episode of every world where its output is true. |
|
Truncates episodes after |
|
Configuration for a command term, which is resampled on reset and whenever its resampling timer expires. |
|
Configuration for an event term, which writes its |
|
When an event runs: once at startup, on reset, or at random intervals. |
|
Configuration for a quantity: a value per world that other terms read as |
|
Noise drawn uniformly from |
|
Select the state |
Building and running#
Collects term uses, lets users edit their order, and compiles them with |
|
An immutable, compiled MDP, created with |
|
Data of an MDP that changes as it runs, such as episode lengths, term state, timers, and random streams. |
|
Results of one step, written into buffers that the caller owns. |
|
Backend that runs the term implementations. |
|
The runtime functions of an MDP, bound to fixed buffers and replayed as CUDA graphs. |
|
Run the |
|
Store the actions, shape |
|
Compute terminations and rewards, resample due commands, run due interval events, and observe. |
|
Reset the masked worlds: restart timers, run reset events, resample commands, then reset term state. |
|
Compute observations for the masked worlds and leave the others unchanged. |
Scene data#
Interface through which an MDP reads and writes scene data. |
|
Description of what a scene provides to an MDP. |
|
Element names and field layouts of one scene entity. |
|
Layout of an entity field, an array of shape |
- class isaaclab_experimental.mdp2.Term[source]#
A reusable calculation with separate state per use. A function is short for
Implementation(compute=fn).Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(torch: ~isaaclab_experimental.mdp2.terms.Implementation | ~collections.abc.Callable | None = None, warp: ~isaaclab_experimental.mdp2.terms.Implementation | ~collections.abc.Callable | None = None, shape: tuple[str | int, ...] = ('worlds', ), state: dict[str, ~isaaclab_experimental.mdp2.terms.State] = <factory>, random: bool = False, init: ~collections.abc.Callable | None = None, capturable: bool = True) None#
- class isaaclab_experimental.mdp2.Implementation[source]#
Backend hooks of a term. Hooks take any of the term’s inputs, state, params, and
rngby name.Fields arrive as arrays of shape
(num_worlds, ...), and subsets asint32index arrays.Torch hooks are batched over worlds:
compute(**args)returns the output. Quantities, terminations, and rewards are computed once per step, so theircomputemay also advance the term’s state in place. An observation’s must not: observations are computed again after a reset, and continuing worlds must get the same sample. Nor may thecomputeof a quantity that observations read, unless repeating it leaves the state as it is.update(**args)advances state in place, once per step aftercompute, for terms whosecomputemust not. A hook that writes a field that other world configs share, such as a command, also takesmask, the worlds where its op runs (None for every world).reset(mask, **args)resets the masked worlds’ state, after the reset events ran and the commands were resampled. Without it, state is set toinitial.
Warp hooks are
wp.funcs that handle one world, called from the kernels generated for the MDP:compute(world, *indices, **args)returns one output element, with one index per output dimension after"worlds";boolfor terminations andfloat32otherwise.update(world, **args)andreset(world, **args)act on one world;resetruns for reset worlds only.A func gets
rngas its world’sint32seed. It must only write its own world, and may read other worlds only where no term writes.
A Python hook, a Torch hook or a Warp hook that launches kernels itself, may also take scene entities bound as inputs, such as
inputs={"robot": robot}, to call any of their methods, andcache, a dict that the use keeps across calls for host objects. A capturable Python hook runs once, at capture, and its launches replay: it must not read values on the host or draw from Torch’s random generators, and its Python branches take the path they took at capture. Mark the termcapturable=Falseotherwise: it then runs on the host between the graphs.A Warp hook can instead be a function that launches kernels itself, with the Torch arguments plus
outforcompute, an array of the output shape to fill for every world, andrngas the stream of every world. It runs between the generated kernels.Event and command terms are
compute(mask, **args), or a funccompute(world, **args)run for the selected worlds, and write their outputs in place. A command gets its value ascommand:computeresamples it and writes the command’s state,updatemay change it every step, and commands have noresethook.Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.State[source]#
A
float32buffer that a term keeps across steps, with one value per world and element by default.Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.TermContext[source]#
Data that an init hook reads and writes, as Torch views over its worlds.
Methods:
entity(argument)Return the scene object, such as an articulation, of an entity input or of the entity an input selects.
__init__(num_worlds, step_dt, inputs, ...)- entity(argument: str) Any[source]#
Return the scene object, such as an articulation, of an entity input or of the entity an input selects.
- __init__(num_worlds: int, step_dt: float, inputs: dict[str, torch.Tensor], outputs: dict[str, torch.Tensor], state: dict[str, torch.Tensor], rng: torch.Tensor | None, selectors: Mapping[str, FieldSelector | SubsetSelector | Entity], scene_data: SceneData) None#
- class isaaclab_experimental.mdp2.Entity[source]#
A scene entity. Accessing an attribute selects one of its fields, as in
robot.joint_vel.Methods:
joints(*patterns)Select the joints whose names match any of the patterns.
bodies(*patterns)Select the bodies whose names match any of the patterns.
__init__(name[, scene])- joints(*patterns: str) SubsetSelector[source]#
Select the joints whose names match any of the patterns.
- bodies(*patterns: str) SubsetSelector[source]#
Select the bodies whose names match any of the patterns.
- class isaaclab_experimental.mdp2.FieldSelector[source]#
Selects a field of a scene entity (
robot.joint_pos) or of the MDP (mdp2.commands.target).Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.SubsetSelector[source]#
Selects the elements of an entity dimension whose names match regular expressions, in entity order.
Bound to an input, as in
inputs={"joint_ids": robot.joints(".*_hip")}, it passes the term anint32index array. The argument name also becomes a dimension that the term’s shapes can use.Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.TaskCfg[source]#
Configuration of the MDP of a task. Setting a term use to
Nonedisables it.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(actions: dict[str, ~isaaclab_experimental.mdp2.config.DirectAction | None] = <factory>, commands: dict[str, ~isaaclab_experimental.mdp2.config.Command | None] = <factory>, quantities: dict[str, ~isaaclab_experimental.mdp2.config.Quantity | None] = <factory>, observations: dict[str, dict[str, ~isaaclab_experimental.mdp2.config.Observation | None]] = <factory>, rewards: dict[str, ~isaaclab_experimental.mdp2.config.Reward | None] = <factory>, terminations: dict[str, ~isaaclab_experimental.mdp2.config.Termination | ~isaaclab_experimental.mdp2.config.TimeLimit | None] = <factory>, events: dict[str, ~isaaclab_experimental.mdp2.config.Event | None] = <factory>) None#
- class isaaclab_experimental.mdp2.DirectAction[source]#
Configuration for an action that writes
scale * action + offsetto an entity field.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(target: FieldSelector, scale: float = 1.0, offset: float | FieldSelector = 0.0, subset: SubsetSelector | None = None) None#
- class isaaclab_experimental.mdp2.Observation[source]#
Configuration for an observation term, whose output is flattened per world into its group’s buffer.
As in MDP1, the noise is added first, then the value is clipped, then scaled. With a history, the group holds the last
history_lengthvalues, oldest first, and a new episode starts with its first value repeated.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, noise: ~isaaclab_experimental.mdp2.config.UniformNoise | None = None, clip: tuple[float, float] | None = None, scale: float | None = None, history_length: int = 0) None#
- class isaaclab_experimental.mdp2.Reward[source]#
Configuration for a reward term, whose output adds
weight * value * step_dtto the reward.Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.Termination[source]#
Configuration for a termination term, which ends the episode of every world where its output is true.
Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.TimeLimit[source]#
Truncates episodes after
episode_length_sseconds.Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.Command[source]#
Configuration for a command term, which is resampled on reset and whenever its resampling timer expires.
Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.Event[source]#
Configuration for an event term, which writes its
outputsfor the worlds selected bymask.Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(term: ~isaaclab_experimental.mdp2.terms.Term, inputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector | ~isaaclab_experimental.mdp2.config.SubsetSelector] = <factory>, params: dict[str, ~typing.Any] = <factory>, outputs: dict[str, ~isaaclab_experimental.mdp2.config.FieldSelector] = <factory>, mode: ~isaaclab_experimental.mdp2.config.EventMode = EventMode.RESET, interval_range_s: tuple[float, float] | None = None, commit: bool | None = None) None#
- class isaaclab_experimental.mdp2.EventMode[source]#
When an event runs: once at startup, on reset, or at random intervals.
- class isaaclab_experimental.mdp2.Quantity[source]#
Configuration for a quantity: a value per world that other terms read as
mdp2.quantities.<name>.Quantities are computed once per step, before the terminations and rewards, so that terms share a calculation instead of repeating it. A quantity keeps its value until the next step: reset events read what the episode that ended left, such as whether it succeeded. A quantity that observations or action offsets read is the exception: it is computed again after a reset, with the quantities that it reads.
Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.UniformNoise[source]#
Noise drawn uniformly from
[n_min, n_max]for each element, like MDP1’sUniformNoiseCfg.Methods:
- classmethod __new__(*args, **kwargs)#
- isaaclab_experimental.mdp2.term_state(use: str, name: str) FieldSelector[source]#
Select the state
nameof another term use, as interm_state("rewards.success", "succeeded").Terms read it as an input; only the use that owns it writes it. Reset events see the state of the episode that ended, since term state resets after them. World configs that define the use differently share its state, each in its own worlds, so its shapes must agree; state that the worlds share, such as a table filled by
init, is filled by each of them in turn.
- class isaaclab_experimental.mdp2.MDPBuilder[source]#
Collects term uses, lets users edit their order, and compiles them with
finalize().Uses run phase by phase. Within a phase they run in the order they were added, unless the order is edited with
move_before(),move_after(), orset_order(). Every edit is checked so that a use that reads a field still runs after the uses of its phase that write it. Action and observation columns follow the schedule order.Methods:
__init__(schema)replicate(builder, num_worlds)Run the task of another builder in the next
num_worldsworlds.add_task(task)Add all term uses of a task in declaration order, skipping uses set to
None.entry(*path)Return the entry at a path, for example
builder.entry("rewards", "alive").schedule()Return all entries in execution order: by phase, then by their order within the phase.
move_before(entry, anchor)Run
entryjust beforeanchor, which must be in the same phase.move_after(entry, anchor)Run
entryjust afteranchor, which must be in the same phase.set_order(phase, entries)Set the execution order of all entries of a phase.
describe()Return the schedule as readable text, listing each use with the fields it reads and writes.
inspect(entry[, backend])Compile the schedule and return the op of an entry, or
Nonefor a time limit.preview_segments([backend])Return how the current order splits each runtime function into CUDA graphs and eager steps.
finalize([device, backend, diagnostics, ...])Validate the schedule and compile it into an immutable MDP.
- __init__(schema: SceneSchema) None[source]#
- replicate(builder: MDPBuilder, num_worlds: int) None[source]#
Run the task of another builder in the next
num_worldsworlds.The builder’s schedule is copied, so later edits to it do not apply. Once worlds are assigned, every world must be, and term uses are added to the replicated builders rather than to this one.
- add_task(task: TaskCfg) None[source]#
Add all term uses of a task in declaration order, skipping uses set to
None.
- entry(*path: str) ScheduleEntry[source]#
Return the entry at a path, for example
builder.entry("rewards", "alive").
- schedule() tuple[ScheduleEntry, ...][source]#
Return all entries in execution order: by phase, then by their order within the phase.
- move_before(entry: ScheduleEntry, anchor: ScheduleEntry) None[source]#
Run
entryjust beforeanchor, which must be in the same phase.
- move_after(entry: ScheduleEntry, anchor: ScheduleEntry) None[source]#
Run
entryjust afteranchor, which must be in the same phase.
- set_order(phase: Phase, entries: list[ScheduleEntry]) None[source]#
Set the execution order of all entries of a phase.
- describe() str[source]#
Return the schedule as readable text, listing each use with the fields it reads and writes.
- inspect(entry: ScheduleEntry, backend: Backend | str = Backend.TORCH) ActionOp | TermOp | None[source]#
Compile the schedule and return the op of an entry, or
Nonefor a time limit.
- preview_segments(backend: Backend | str = Backend.TORCH) dict[str, tuple[Segment, ...]][source]#
Return how the current order splits each runtime function into CUDA graphs and eager steps.
- finalize(device: str = 'cpu', backend: Backend | str = Backend.TORCH, diagnostics: bool = False, final_observations: bool = False) MDP[source]#
Validate the schedule and compile it into an immutable MDP.
Raises a single
ValueErrorthat lists every error found. Editing the builder afterwards does not change the returned MDP. Withdiagnostics,evaluate()also writes the reward and termination flag of each term. Withfinal_observations, it also writes the observations of the worlds that ended, before their reset, toTransition.final_observations, in the kernels that compute them.
- class isaaclab_experimental.mdp2.MDP[source]#
An immutable, compiled MDP, created with
MDPBuilder.finalize().Attributes:
Action counts of the world configs with fewer actions than the widest.
Ops of every phase except actions, in schedule order.
Names of all ops in schedule order.
runs of steps that are captured as one CUDA graph, or run eagerly.
Whether every runtime function can be captured as a single CUDA graph.
Scene fields that the startup events write, which the scene commits after startup.
Scene fields that the reset events write, which the scene commits after a reset.
Scene fields that the interval events write, which the scene commits in the worlds where one ran.
Methods:
describe()Return the compiled MDP as readable text: ops by phase with their layout, then each function's steps.
state(scene_data[, seed])Create the runtime state, bound to the data views of a scene with the same schema.
Allocate one zeroed observation buffer per group, of shape
(num_worlds, group_dim).Allocate the zeroed buffers of a transition, with the optional buffers that the MDP writes.
__init__(schema, device, backend, ops, plan, ...)- property narrow_action_dims: tuple[int, ...][source]#
Action counts of the world configs with fewer actions than the widest.
- property term_ops: tuple[TermOp, ...][source]#
Ops of every phase except actions, in schedule order.
- property segments: dict[str, tuple[Segment, ...]][source]#
runs of steps that are captured as one CUDA graph, or run eagerly.
The Warp backend may need fewer graphs: it runs a hook early when nothing between depends on it.
- Type:
Segments of each runtime function
- property is_fully_capturable: bool[source]#
Whether every runtime function can be captured as a single CUDA graph. Startup runs once, uncaptured.
- property startup_writes: frozenset[FieldSelector][source]#
Scene fields that the startup events write, which the scene commits after startup.
- property reset_writes: frozenset[FieldSelector][source]#
Scene fields that the reset events write, which the scene commits after a reset.
- property interval_writes: frozenset[FieldSelector][source]#
Scene fields that the interval events write, which the scene commits in the worlds where one ran.
- describe() str[source]#
Return the compiled MDP as readable text: ops by phase with their layout, then each function’s steps.
- state(scene_data: SceneData, seed: int = 0) MDPState[source]#
Create the runtime state, bound to the data views of a scene with the same schema.
- observations() dict[str, ProxyArray][source]#
Allocate one zeroed observation buffer per group, of shape
(num_worlds, group_dim).
- transition() Transition[source]#
Allocate the zeroed buffers of a transition, with the optional buffers that the MDP writes.
- __init__(schema: SceneSchema, device: str, backend: Backend, ops: Mapping[Phase, tuple[ActionOp | TermOp, ...]], plan: ExecutionPlan, action_dim: int, observation_dims: Mapping[str, int], max_episode_length: int | None, streams: tuple[str, ...], subsets: Mapping[SubsetSelector, tuple[int, ...]], diagnostics: bool, final_observations: bool, world_configs: tuple[WorldConfig, ...], launches: Launches | None = None) None#
- class isaaclab_experimental.mdp2.MDPState[source]#
Data of an MDP that changes as it runs, such as episode lengths, term state, timers, and random streams.
Every array is in
arrays, under the key that the backends use for it:a field or subset selector: the selected fields of the scene, the fields of the MDP (
mdp2.last_action,mdp2.commands.<name>, …), and the state of every term use,mdp2.term_state(use, name);(mdp2.last_action, action_dim): the first columns of the last actions, which the terms of a world config withaction_dimactions read if other world configs have more;"rng": the random streams,(num_streams, num_worlds), seerng;"reward_weights":(num_worlds, num_rewards), which may change while the MDP runs, also in a CUDA graph;("timer", op): seconds left per world, for a timed op;("history", op):(num_worlds, history_length, size), for an observation with a history;("scratch", op): the output of a Python hook in a Warp MDP;("selected", op): the worlds that the hook of a timed op, or of an op active in some worlds, acts on;("active", op): the worlds where an op is active, for an op that is active in only some;"interval_worlds": the worlds where an interval event that writes scene fields ran in the lastevaluate, for an MDP with such events.
Attributes:
Control steps since the last reset,
(num_worlds,).The last actions,
(num_worlds, action_dim).Methods:
__init__(arrays, arguments, scene_data[, caches])- property episode_length: ProxyArray[source]#
Control steps since the last reset,
(num_worlds,).
- property actions: ProxyArray[source]#
The last actions,
(num_worlds, action_dim).
- class isaaclab_experimental.mdp2.Transition[source]#
Results of one step, written into buffers that the caller owns.
Methods:
- classmethod __new__(*args, **kwargs)#
- __init__(rewards: ProxyArray, terminated: ProxyArray, truncated: ProxyArray, done: ProxyArray, observations: dict[str, ProxyArray], reward_terms: ProxyArray | None = None, termination_terms: ProxyArray | None = None, final_observations: dict[str, ProxyArray] | None = None) None#
- class isaaclab_experimental.mdp2.CapturedMDP[source]#
The runtime functions of an MDP, bound to fixed buffers and replayed as CUDA graphs.
Consecutive capturable launches of a function are captured as one graph, and hooks that cannot be captured run eagerly between the graphs. Capturing runs nothing. Before each call, write the actions and the mask of the worlds to reset or observe into the bound
actionsandmaskbuffers. On CPU, the launches run eagerly.Methods:
__init__(mdp, state, transition, actions, mask)Store the bound actions and write them to their target fields.
evaluate()Compute terminations, rewards, and successor observations into the bound transition.
reset()Reset the worlds selected by the bound mask.
observe()Compute the observations of the worlds selected by the bound mask into the bound transition.
- __init__(mdp: MDP, state: MDPState, transition: Transition, actions: ProxyArray, mask: ProxyArray, commit: Callable[[], None] | None = None) None[source]#
- isaaclab_experimental.mdp2.startup(mdp: MDP, state: MDPState) None[source]#
Run the
inithooks of the terms, then the startup events, once, in every world.
- isaaclab_experimental.mdp2.apply_actions(mdp: MDP, state: MDPState, actions: ProxyArray) None[source]#
Store the actions, shape
(num_worlds, action_dim), and write them to their target fields.
- isaaclab_experimental.mdp2.evaluate(mdp: MDP, state: MDPState, transition: Transition) None[source]#
Compute terminations and rewards, resample due commands, run due interval events, and observe.
- isaaclab_experimental.mdp2.reset(mdp: MDP, state: MDPState, mask: ProxyArray, commit: Callable[[], None] | None = None) None[source]#
Reset the masked worlds: restart timers, run reset events, resample commands, then reset term state.
commitmakes the scene see the writes of the reset events so far. It is called before every event that asks for it, and tasks without such events need none. The caller commits the writes once more after the reset.
- isaaclab_experimental.mdp2.observe(mdp: MDP, state: MDPState, observations: dict[str, ProxyArray], mask: ProxyArray) None[source]#
Compute observations for the masked worlds and leave the others unchanged.
Recomputes the quantities that observations read, and fills the observation histories of new episodes, in the masked worlds. Changes no other MDP state.
- class isaaclab_experimental.mdp2.SceneData[source]#
Interface through which an MDP reads and writes scene data. The environment owns the scene.
Methods:
data(entity, field)Return a live view of an entity field, of shape
(num_worlds, ...).entity(name)Return the scene object of an entity, such as an articulation, for Python hooks that take it.
commit(mask, fields)Make the simulation see the
fieldsthat events wrote in the masked worlds, and refresh derived data.__init__(*args, **kwargs)- data(entity: str, field: str) ProxyArray[source]#
Return a live view of an entity field, of shape
(num_worlds, ...).
- entity(name: str) Any[source]#
Return the scene object of an entity, such as an articulation, for Python hooks that take it.
- commit(mask: array, fields: Collection[FieldSelector]) None[source]#
Make the simulation see the
fieldsthat events wrote in the masked worlds, and refresh derived data.
- __init__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.SceneSchema[source]#
Description of what a scene provides to an MDP. The builder resolves selectors against it.
Methods:
- classmethod __new__(*args, **kwargs)#
- class isaaclab_experimental.mdp2.EntitySchema[source]#
Element names and field layouts of one scene entity.
Methods:
- classmethod __new__(*args, **kwargs)#