Franka Cube Stacking with MimicGen

This example demonstrates the AutoData workflow using MimicGen to generate a synthetic dataset for a Franka robot performing a cube-stacking task. The workflow covers recording source demonstrations by teleoperation, annotating their subtask boundaries, generating a large dataset with MimicGen, and validating the result.

The Franka cube-stacking task.

Task Overview

Environment name: Isaac-Stack-Cube-Franka-IK-Rel-v0

Task Description: A Franka arm stacks three cubes on a table — red on blue, then green on red.

Key Specifications:

Property	Value
Algorithm	MimicGen (single end-effector)
Embodiment	Franka, relative IK task-space actions (7-D: delta pose (xyz, rpy) + binary gripper open/close)
Task descriptor	franka_cube_stack.yaml
Embodiment config	franka_ik_rel.yaml
Subtasks	Grasp red cube (grasp_1) → stack red on blue (stack_1) → grasp green cube (grasp_2) → place green on red (end of trajectory)
Pre-annotated source dataset	datasets/annotated_datasets/dataset_franka_annotated.hdf5

Workflow

The tutorial covers the pipeline to go from a handful of human demonstrations to a large synthetically generated dataset ready for policy training. You can follow the whole pipeline, or skip the first two steps by using the pre-annotated dataset that ships with the repository.

Prerequisites

Start the dev container (see Installation):

./docker/run_docker.sh

Workflow Steps

Follow these steps to complete the workflow:

• Step 1: Record Source Demonstrations

• Step 2: Annotate Demonstrations

• Step 3: Generate the Dataset

• Step 4: Validate the Generated Dataset