Reinforcement Learning Library Comparison

In this section, we provide an overview of the supported reinforcement learning libraries in Isaac Lab, along with performance benchmarks across the libraries.

The supported libraries are:

• SKRL

• RSL-RL

• RL-Games

• Stable-Baselines3

Feature Comparison

Feature	RL-Games	RSL RL	SKRL	Stable Baselines3
Algorithms Included	PPO, SAC, A2C	PPO	Extensive List	Extensive List
Vectorized Training	Yes	Yes	Yes	No
Distributed Training	Yes	No	Yes	No
ML Frameworks Supported	PyTorch	PyTorch	PyTorch, JAX	PyTorch
Multi-Agent Support	PPO	PPO	PPO + Multi-Agent algorithms	External projects support
Documentation	Low	Low	Comprehensive	Extensive
Community Support	Small Community	Small Community	Small Community	Large Community
Available Examples in Isaac Lab	Large	Large	Large	Small

Training Performance

We performed training with each RL library on the same Isaac-Humanoid-v0 environment with --headless on a single RTX 4090 GPU and logged the total training time for 65.5M steps for each RL library.

RL Library	Time in seconds
RL-Games	216
RSL RL	215
SKRL	321
Stable-Baselines3	6320