LeRobot v0.5.0: Scaling Every Dimension

## LeRobot v0.5.0: Scaling Every Dimension

The LeRobot project continues its steady progress, and this v0.5.0 release showcases remarkable changes compared to previous versions. Built with over 200 PR merges and contributions from more than 50 new contributors, LeRobot v0.5.0 has achieved explosive growth and development across all areas – hardware, policies, datasets, simulation environments, and the coding base. This release particularly expands the **LeRobot** ecosystem and provides the foundation for training and deploying policies on both real hardware and simulation environments.

### Key Changes Summary (TL;DR)

LeRobot v0.5.0 introduces full Unitree G1 humanoid support (whole-body control model), new policies including Pi0-FAST self-regressive VLA and real-time chunking (RTC) for real-time inference, and streaming video encoding to eliminate wait times between recorded episodes. It also includes EnvHub for loading simulation environments from Hugging Face Hub, NVIDIA IsaacLab-Arena integration, Python 3.12+, Transformers v5, a major coding base modernization with third-party policy plugins.

### Expanded Hardware Support: More Robots

LeRobot v0.5.0 significantly expands the range of supported devices from existing arm robots and mobile robots to full-body humanoid robots. This is a significant advancement for the **LeRobot** project, moving from tabletop arm robots to full-body embedded AI.

#### Unitree G1 Humanoid Support

The most significant hardware addition in this release is the Unitree G1 humanoid support. This is the first humanoid integration for LeRobot and provides the following features:

* **Walking and Locomotion:** Navigates and moves through the environment.
* **Manipulation:** Performs tasks involving skillful manipulation of objects.
* **Remote Control:** Uses an intuitive remote control interface to remotely control the G1.
* **Whole-Body Control (WBC):** Simultaneously coordinates walking and manipulation for complex, real-world tasks.

#### OpenArm and OpenArm Mini Support

Support for OpenArm robots and OpenArm Mini teleoperators has also been added. OpenArm is a powerful arm robot seamlessly integrated with LeRobot, and the Mini serves as a natural teleoperation device. Both support dual-arm setups for performing more complex manipulation tasks.

#### Additional Robot Support

Earth Rover: The first mobile robot integration that brings LeRobot to outdoor exploration and ground robotics.

OMX Robot: A new arm robot with configurable gripper setups and calibration support.

SO-100/SO-101 Integration: Integrates SO-100 and SO-101 implementations into a single, clean codebase, reducing code duplication and facilitating maintenance.

#### CAN Bus Motor Support

New motor controller support through CAN (Controller Area Network) bus support leads to higher-performance actuators.

RobStride: A CAN-based motor controller for high-torque applications.

Damiao: Another CAN bus motor controller expands the range of compatible hardware.

### Policies: A Growing Model Zoo

This release adds six new policies and technologies to LeRobot, expanding the boundaries of open-source robot learning. By supporting a variety of policies and technologies, **LeRobot** increases its versatility.

#### Pi0-FAST: Self-Regressive VLA

Pi0-FAST introduces self-regressive Vision-Language-Action models to LeRobot using FAST (Frequency-space Action Sequence Tokenization). Unlike Pi0’s flow-matching approach, Pi0-FAST uses a self-regressive action expert based on Gemma 300M to generate tokenized actions.

#### Real-Time Chunking (RTC)

Real-time chunking is an inference-time technique borrowed from Physical Intelligence, significantly improving the responsiveness of flow-matching policies. Instead of re-planning when the entire action chunk is complete, RTC continuously mixes new predictions with the ongoing task, creating smoother and more reactive movements.

### Datasets: Faster Recording, Faster Training

This release includes significant performance improvements to the data pipeline, resulting in significantly faster data collection and training. **LeRobot** enhances the efficiency of the dataset pipeline, shortening development time.

#### Streaming Video Encoding

Previously, video encoding had to complete before an episode could be finished. Now, streaming video encoding encodes frames in real-time as the episode is captured, eliminating wait times between episodes.

#### 10x Faster Image Training, 3x Faster Encoding

Improved image training speed by 10x and encoding speed by 3x by addressing data access bottlenecks and overall improving image processing.

#### New Dataset Tools

The dataset editing toolkit continues to grow, offering a variety of features, including subtask support, image-video conversions, new editing tasks, and modifications to existing tasks.

### EnvHub: Bring Environments from the Hub

EnvHub is a new way to use simulation environments in LeRobot. You can load environments directly from the Hugging Face Hub. This feature allows **LeRobot** users to simply point to a Hub repository to have it downloaded, registered to Gymnasium, and ready for training and evaluation.

### Conclusion

This LeRobot v0.5.0 release is more than just a set of new features; it signifies an innovation across the entire robotics learning ecosystem. Expanded robot support, faster training, convenient environment management, and a modernized coding base have further evolved LeRobot into a more powerful and user-friendly platform. The LeRobot team will continue to evolve through active community participation and feedback moving forward.

In-depth Analysis and Implications

Array

Original Source: LeRobot v0.5.0: Scaling Every Dimension