Renderers

Renderers#

Isaac Lab uses a pluggable renderer architecture to support different rendering backends for camera sensors. The BaseRenderer abstract base class defines the interface that all renderer implementations must follow.

Isaac Lab supports three rendering backends:

  • Isaac RTX renderer (IsaacRtxRendererCfg) — NVIDIA’s Omniverse RTX rendering pipeline running inside Isaac Sim. Requires Isaac Sim. Best for photorealistic rendering, full camera sensor support (RGB, depth, semantic segmentation, etc.), and production quality outputs.

  • OVRTX renderer (OVRTXRendererCfg) — A standalone RTX path-tracing renderer provided by the isaaclab_ov extension. Delivers RTX-quality rendering.

  • Newton Warp renderer (NewtonWarpRendererCfg) — A lightweight GPU-accelerated renderer built on NVIDIA Warp. Works with the Newton physics backend and does not require Isaac Sim (kit-less mode). Ideal for training workflows where full RTX fidelity is not needed.

Choosing a renderer backend#

Backend

Requires Isaac Sim?

Best For

Isaac RTX

Yes

Full sensor fidelity, RTX photorealism, PhysX backend

OVRTX

No (kit-less; needs isaaclab_ov + ovrtx)

RTX-quality rendering without requiring Isaac Sim

Newton Warp

No (kit-less)

Newton backend, fast training

Note

Visualization markers are not yet supported by Newton-based renderer backends, including the Newton Warp renderer. Use an RTX-based renderer, such as the Isaac RTX renderer or OVRTX renderer, when marker visualization is needed.

Note

Temporal information for camera-based RL. Unlike RTX modes with temporal anti-aliasing (DLSS, DLAA, TAA), the Newton Warp renderer does not inject prior-frame information into the current image. Camera-control tasks that depend on velocity-like visual cues should add explicit temporal observations (e.g. task-local frame stacking) rather than relying on renderer-specific artifacts.

Architecture Overview#

The renderer system consists of:

  1. BaseRenderer — Abstract base class defining the rendering lifecycle and interface

  2. Renderer — Factory that instantiates the appropriate backend based on renderer configuration class

  3. RendererCfg — Base configuration; each backend extends it with backend-specific options

  4. Concrete implementations — Backend-specific renderers in extension packages

  5. RenderContext — A management class for instantiating and accessing renderer instances using a RendererCfg. After instantiation, a config can then be used to acquire the instance of the renderer as needed.

import isaaclab.sim as sim_utils
from isaaclab.renderers import BaseRenderer
from isaaclab_newton.renderers import NewtonWarpRendererCfg

# Create a Newton Warp renderer (no Isaac Sim required)
sim_ctx = sim_utils.SimulationContext.instance()
# RenderContext.get_renderer will instantiate the renderer backend
# or return an existing renderer with a matching config
renderer: BaseRenderer = sim_ctx.render_context.get_renderer(NewtonWarpRendererCfg())
assert isinstance(renderer, BaseRenderer)

For the RTX renderer (requires Isaac Sim):

import isaaclab.sim as sim_utils
from isaaclab.renderers import BaseRenderer
from isaaclab_physx.renderers import IsaacRtxRendererCfg

# Create an RTX renderer
sim_ctx = sim_utils.SimulationContext.instance()
# RenderContext.get_renderer will instantiate the renderer backend
# or return an existing renderer with a matching config
renderer: BaseRenderer = sim_ctx.render_context.get_renderer(IsaacRtxRendererCfg())

For RTX renderer settings and presets (quality, balanced, performance), see Configuring RTX Rendering Settings.

Core concepts#

  • Use the RenderContext: Always instantiate renderers via the RenderContext with a renderer-specific config class (e.g. sim_ctx.render_context.get_renderer(IsaacRtxRendererCfg())). Do not import or instantiate concrete backend classes (e.g. IsaacRtxRenderer, OVRTXRenderer) directly—their names and package locations are implementation details and may change without notice.

  • Lightweight config imports: Importing a renderer configuration class does not pull in backend-specific dependencies. The backend is lazily loaded when the renderer is instantiated, and instantiation may fail if the backend is not installed.

    import isaaclab.sim as sim_utils
from isaaclab.renderers import BaseRenderer
# Lightweight: does not import OVRTX backend dependencies
from isaaclab_ov.renderers import OVRTXRendererCfg

# Lazily loads ovrtx when instantiated; may fail if isaaclab_ov / ovrtx is not installed
sim_ctx = sim_utils.SimulationContext.instance()
renderer: BaseRenderer = sim_ctx.render_context.get_renderer(OVRTXRendererCfg())

Installing the OVRTX renderer#

The OVRTX renderer is provided by the isaaclab_ov extension. The extension’s source package ships with the core install, but the renderer’s ovrtx runtime wheel (the ovrtx package, hosted on pypi.nvidia.com) is not installed by default. You must request it explicitly — OVRTX does not require Isaac Sim.

Install via the Isaac Lab CLI using the ov[ovrtx] token:

# Install the ovrtx runtime wheel on top of an existing install
./isaaclab.sh -i ov[ovrtx]

Note

The bare ov token does not install any runtime wheel (the source packages are already part of the core install). Use ov[ovrtx] (or ov[all]) to pull in the ovrtx dependency.

Or install manually with pip (note the [ovrtx] extra and the extra index URL):

pip install --extra-index-url https://pypi.nvidia.com -e "source/isaaclab_ov[ovrtx]"

  • Opaque render data: The render data object returned by create_render_data() is passed to subsequent renderer methods. It should be completely opaque to the caller: inspecting or modifying it via get/set attributes is an anti-pattern and breaks the API contract.

Note

The BaseRenderer class is under active development and may change without notice.

See Also#