Reinforcement Learning Library Comparison#
In this section, we provide an overview of the supported reinforcement learning libraries in Isaac Lab, along with performance benchmarks across the libraries.
The supported libraries are:
Feature Comparison#
Feature |
RL-Games |
RSL RL |
SKRL |
Stable Baselines3 |
---|---|---|---|---|
Algorithms Included |
PPO, SAC, A2C |
PPO |
||
Vectorized Training |
Yes |
Yes |
Yes |
No |
Distributed Training |
Yes |
No |
Yes |
No |
ML Frameworks Supported |
PyTorch |
PyTorch |
PyTorch, JAX |
PyTorch |
Multi-Agent Support |
PPO |
PPO |
PPO + Multi-Agent algorithms |
External projects support |
Documentation |
Low |
Low |
Comprehensive |
Extensive |
Community Support |
Small Community |
Small Community |
Small Community |
Large Community |
Available Examples in Isaac Lab |
Large |
Large |
Large |
Small |
Training Performance#
We performed training with each RL library on the same Isaac-Humanoid-v0
environment
with --headless
on a single RTX 4090 GPU
and logged the total training time for 65.5M steps for each RL library.
RL Library |
Time in seconds |
---|---|
RL-Games |
216 |
RSL RL |
215 |
SKRL |
321 |
Stable-Baselines3 |
6320 |