Setting up Haply Teleoperation

Setting up Haply Teleoperation#

The Haply Inverse3 and VerseGrip let you command an end-effector pose and feel contact forces from the simulation. This guide connects Haply Devices to Isaac Lab for haptic teleoperation.

Overview#

Using Haply with Isaac Lab involves the following components:

  • Isaac Lab simulates the robot environment and streams contact forces back to the operator

  • Haply Inverse3 provides 3-DOF position tracking and force feedback in the operator’s workspace

  • Haply VerseGrip adds orientation sensing and button inputs for gripper control

  • Haply SDK manages WebSocket communication between Isaac Lab and the Haply hardware

This guide will walk you through:

System Requirements#

Hardware Requirements#

  • Isaac Lab Workstation

    • Ubuntu 22.04 or Ubuntu 24.04

    • Hardware requirements for 200Hz physics simulation:

      • CPU: 8-Core Intel Core i7 or AMD Ryzen 7 (or higher)

      • Memory: 32GB RAM (64GB recommended)

      • GPU: RTX 3090 or higher

    • Network: Same local network as Haply devices for WebSocket communication

  • Haply Devices

    • Haply Inverse3 - Haptic device for position tracking and force feedback

    • Haply VerseGrip - Wireless controller for orientation and button inputs

    • Both devices must be powered on and connected to the Haply SDK

Software Requirements#

  • Isaac Lab (follow the installation guide)

  • Haply SDK (provided by Haply Robotics)

  • Python 3.12+

  • websockets Python package (automatically installed with Isaac Lab)

Installation#

1. Install Isaac Lab#

Follow the Isaac Lab installation guide to set up your environment.

The websockets dependency is automatically included in Isaac Lab’s requirements.

2. Install Haply SDK#

Download the Haply SDK from the Haply Devices website. Install the SDK software and configure the devices.

3. Verify Installation#

Test that your Haply devices are detected by the Haply Device Manager. You should see both Inverse3 and VerseGrip as connected.

Device Setup#

1. Physical Setup#

  • Place the Haply Inverse3 on a stable surface

  • Ensure the VerseGrip is charged and paired

  • Position yourself comfortably to reach the Inverse3 workspace

  • Keep the workspace clear of obstacles

2. Start Haply SDK#

Launch the Haply SDK according to Haply’s documentation. The SDK typically:

  • Runs a WebSocket server on localhost:10001

  • Streams device data at 200Hz

  • Displays connection status for both devices

3. Test Communication#

You can test the WebSocket connection using the following Python script:

import asyncio
import websockets
import json

async def test_haply():
    uri = "ws://localhost:10001"
    async with websockets.connect(uri) as ws:
        response = await ws.recv()
        data = json.loads(response)
        print("Inverse3:", data.get("inverse3", []))
        print("VerseGrip:", data.get("wireless_verse_grip", []))

asyncio.run(test_haply())

You should see device data streaming from both Inverse3 and VerseGrip.

Running the Example#

The Haply teleoperation example showcases robot manipulation with force feedback using a Franka Panda arm.

Basic Usage#

# Ensure Haply SDK is running
isaaclab example haply-teleoperation --websocket_uri ws://localhost:10001 --pos_sensitivity 1.65

The example will:

  1. Connect to the Haply devices via WebSocket

  2. Spawn a Franka Panda robot and a cube in simulation

  3. Map Haply position to robot end-effector position

  4. Stream contact forces back to the Inverse3 for haptic feedback

Controls#

  • Move Inverse3: Controls the robot end-effector position

  • VerseGrip Button A: Open gripper

  • VerseGrip Button B: Close gripper

  • VerseGrip Button C: Rotate end-effector by 60°

Advanced Options#

Customize the example with command-line arguments:

# Use custom WebSocket URI
isaaclab example haply-teleoperation --websocket_uri ws://192.168.1.100:10001

# Adjust position sensitivity (default: 1.0)
isaaclab example haply-teleoperation --websocket_uri ws://localhost:10001 --pos_sensitivity 2.0

Example Features#

  • Workspace Mapping: Haply workspace is mapped to robot reachable space with safety limits

  • Inverse Kinematics: Inverse Kinematics (IK) computes joint positions for desired end-effector pose

  • Force Feedback: Contact forces from end-effector sensors are sent to Inverse3 for haptic feedback

Troubleshooting#

No Haptic Feedback#

Problem: No haptic feedback felt on Inverse3

Solutions:

  • Verify Inverse3 is the active device in Haply SDK

  • Check contact forces are non-zero in simulation (try grasping the cube)

  • Ensure limit_force is not set too low (default: 2.0N)

Next Steps#

  • Customize the example: Modify the workspace mapping or add custom button behaviors

  • Implement your own controller: Use HaplyDevice in your own scripts

For more information on device APIs, see HaplyDevice in the API documentation.