Welcome to Isaac Lab-Arena!#
Note
This is the development version of Isaac Lab Arena. It contains the newest features but may not be fully tested yet. For the tested version, please refer to the release/0.3.0 branch.
Overview
NVIDIA Isaac Lab-Arena is an open-source framework for scalable benchmark authoring and robot policy evaluation in simulation.
Not another benchmark or library of benchmarks. Arena extends Isaac Lab with reusable APIs to author robot policy evaluations, execute at scale, and analyze results for actionable feedback.
Features#
Author benchmarks#
Modular environment composition
Build dynamic, LEGO-like environments from independent scene, object, embodiment, and task modules. Swap one without rebuilding the rest.
Scene
Kitchen
Objects
Banana + plate
Embodiment
Franka
Task
Pick → place
ArenaEnvBuilder
Environment family
Define layouts with semantic spatial relations rather than hand-coded poses. Arena solves and validates candidate placements against object geometry, collisions, and task constraints.
Before Arena
Hard-coded coordinates
microwave
x 0.82 · y −0.14 · z 0.91
→
With Arena
Describe spatial intent
plate On table
banana On plate
→
Arena resolves + builds
Intent becomes a valid simulation environment
Placement solver
→
Environment ready for evaluation
Agentic environment generation
Experimental? The underlying agents and architecture may change in future versions.Describe the benchmark you want in natural language. The agent infers constraints, produces an editable spec, fetches existing SimReady USDs, and builds the Arena environment(s).
Input prompt
Using a DROID robot and maple table, pick up one fruit, one bottle, and one utensil in any order, then place all three into a bin. Generate each environment with a different fruit, bottle, utensil, and bin. Add distractor kitchen objects, such as a mug and a SPAM can, to the table; the distractors are not part of the task.
→
Output: Ready-to-evaluate Arena environments
Turn one environment into a controlled sweep of conditions. Define ranges and distributions once; sample them at build time or reset.
HDR background
Light color
Color temperature
Light direction
Isaac Lab interoperability
Author benchmarks and execute policy evaluations in Arena. Register the Arena-authored environment in Isaac Lab with ease to perform data collection or policy learning in Isaac Lab.
Execute large-scale parallel policy evaluations#
Large-scale parallel environments
Evaluate one policy concurrently across thousands of heterogeneous (object-level) environments instead of sequential rollouts to speed up policy evaluation. Lightwheel case study reports 10x faster despite using higher fidelity assets. A preliminary camera-free scaling benchmark measured 2,390 environment-steps/s of rollout throughput with 1,024 parallel environments on one RTX 5880 Ada Generation GPU.
Large-scale multi-node evaluation
Define experiments with multiple policies and tasks. Run them locally or use an orchestrator such as OSMO to distribute across multi-node compute. Arena returns aggregate metrics for a high-level summary and per-episode results for detailed analysis. A preliminary OSMO scaling benchmark showed that active execution time decreased nearly in proportion to the number of GPUs used to run independent Runs concurrently.
Define experiment
- Tasks
- 20 RoboLab tasks
- Policies
- Policy A · Policy B
- Experiment
- 100 episodes / task
→
Run evaluations
Local or distributed compute
→
Collate results
One combined result
Run leading foundation models GR00T, π0.5, or bring your own policy behind a server. Arena exchanges observations and actions through one client contract across processes, GPUs, or machines.
Analyze robustness and why policies fail#
Subtask predicates · The where
Track fine-grained milestones within each subtask—for example, decomposing pick-and-place into grasp, lift, transport, and place—to see where a policy fails.
Before Arena — End-to-end result only. Success: 20%. Failure stage: unknown.
After Arena · predicates — Track progress within every rollout and see where success drops.
Grasp
30.3%
Lift
22.9%
Transport
21.3%
Place
20.0%
Failure stage: Lift. 75 rollouts grasp the object but never lift it.
Sensitivity analysis · The why
Perturb environment factors to reveal policy robustness and get actionable feedback that can inform the next round of targeted policy learning.
Before Arena — End-to-end result only. Success: 20%. Root cause: unknown.
After Arena · sensitivity analysis — View which factors the policy is sensitive to. The same failure can have different root causes and require different fixes.
The failure cliff identifies a robustness gap and enables targeted policy improvement.
The failure cliff identifies a robustness gap and enables targeted policy improvement.
Why Isaac Lab-Arena#
Simulation can provide early feedback before expensive real-world deployment.
But as policy evaluation expands across tasks, scenes, conditions, and robot embodiments,standalone environments, custom scripts, and sequential runs do not scale.
Support#
GitHub Discussions
GitHub Issues
Omniverse Discord
NVIDIA Forums
Project resources#
Contribution workflow
Publishing workflow
Citation guidance
Apache 2.0 license
Acknowledgements#
Isaac Lab-Arena builds on NVIDIA Isaac Lab, with the evaluation and task layers designed in close collaboration with Lightwheel. We thank the Isaac Lab team and the broader robotics community for their foundational work.
Isaac Lab-Arena was built in collaboration with the authors of Robolab (website, paper).