Reinforcement Learning Library Comparison

Reinforcement Learning Library Comparison#

In this section, we provide an overview of the supported reinforcement learning libraries in Isaac Lab, along with performance benchmarks across the libraries.

The supported libraries are:

Feature Comparison#

Feature

RL-Games

RSL RL

SKRL

Stable Baselines3

Algorithms Included

PPO, SAC, A2C

PPO

Extensive List

Extensive List

Vectorized Training

Yes

Yes

Yes

No

Distributed Training

Yes

No

Yes

No

ML Frameworks Supported

PyTorch

PyTorch

PyTorch, JAX

PyTorch

Multi-Agent Support

PPO

PPO

PPO + Multi-Agent algorithms

External projects support

Documentation

Low

Low

Comprehensive

Extensive

Community Support

Small Community

Small Community

Small Community

Large Community

Available Examples in Isaac Lab

Large

Large

Large

Small

Training Performance#

We performed training with each RL library on the same Isaac-Humanoid-v0 environment with --headless on a single RTX 4090 GPU and logged the total training time for 65.5M steps for each RL library.

RL Library

Time in seconds

RL-Games

216

RSL RL

215

SKRL

321

Stable-Baselines3

6320