Author a Workbench Environment with a Coding Agent#

This workflow uses a coding agent to author and validate a workbench cleanup task.

Prerequisites#

Complete the Arena installation, then set up either the native uv environment or the Arena Docker container:

Prepare the Arena runtime from the repository root, using either a native uv environment or the base Docker container (see Installation for the full setup):

uv sync --extra dev
source .venv/bin/activate
export OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y

./docker/run_docker.sh

Install and authenticate your favourite coding agent on the host, then start it from the repository root. In our example, we use Codex with GPT-6 Astra and high reasoning effort.

Install the Codex CLI on the host using the official installation guide, then authenticate on the host with codex login. Codex CLI 0.160.0 or newer is recommended, check with codex --version.

Step 1: Viewing an Existing Agent-Authored Environment#

Run this step in the Arena runtime (uv environment or inside the Docker container).

Arena includes an agent-generated workbench scene and task:

  • isaaclab_arena_environments/workbench_cleanup/scene_tool_bin_scope.yaml

  • isaaclab_arena_environments/workbench_cleanup/task_full_cleanup.yaml

The scene contains two tools in the open top drawer, an open socket set, an oscilloscope, and a white bin. Preview it in Kit:

Arena runtime (activated uv environment or Arena Docker shell)#
python isaaclab_arena/evaluation/policy_runner.py \
  --env_spec isaaclab_arena_environments/workbench_cleanup/task_full_cleanup.yaml \
  --robot_control droid_differential_ik \
  --policy_type zero_action \
  --num_steps 1000 \
  --viz kit

You should see the workbench scene with objects and the DROID robot holding its pose. The preview exits after 1,000 steps. This preview does not attempt the task.

Next, run the recipe with the Object Control Runner to see a force-based demonstration of all four subtasks:

Arena runtime (activated uv environment or Arena Docker shell)#
python isaaclab_arena/agentic_tools/object_control_runner.py \
  --env_spec isaaclab_arena_environments/workbench_cleanup/task_full_cleanup.yaml \
  --commands isaaclab_arena_environments/workbench_cleanup/object_control/full_cleanup.jsonl \
  --output $HOME/eval/workbench/full_cleanup_demo \
  --seed 42 \
  --placement_seed 42 \
  --visualizer kit

On success, the runner prints the following summary:

TASK_SUCCESS {"success": [true], "report": "/home/<user>/eval/workbench/full_cleanup_demo/validation.json"}

The recipe applies bounded forces and joint efforts to close the drawer and toolcase, place the bin on the overhead shelf, and press the oscilloscope power button. The agentic authoring pipeline leverages this tool to validate the object physics and checks for task success without invoking a robot policy.

Complete all four workbench cleanup subtasks.

Step 2: Authoring a New Workbench Environment#

Start Codex from the repository root on the host:

Host terminal (repository root)#
codex --model gpt-6-astra -c 'model_reasoning_effort="high"' --yolo

Note

The command above will start Codex in yolo mode, which means that the agent runs commands without asking for approval and outside Codex’s sandbox. Do this at your own risk.

Enter a prompt like the one below:

Prompt in the host Codex session#
Set up an environment for evaluation of a workbench cleanup task.
Use the simready_central workbench assets registered in
isaaclab_arena/assets/background_library.py and
isaaclab_arena/assets/object_library.py.

On the tabletop of the workbench, there is a clutter of five tools, such as
wrenches and screwdrivers, at the center. The top drawer of the workbench is
fully open.

Task: put the tools into the drawer one by one, then close the drawer.

Validate task success correctly with a force-based demo script.

The agent uses Arena’s author-environment skill to author and validate the requested environment. After Codex finishes (expect 10~20 minutes), it should return a successful force-based demonstration video like the one below:

Successful five-tool drawer cleanup using bounded forces and joint efforts.