Opening article
Hugging Face has released LeRobotDataset:v3, a new format that packs multiple episodes into single files and supports streaming access. The update aims to overcome file‑system limits when scaling to millions of episodes and integrates with the lerobot library. A conversion script is available to migrate existing v2 datasets to v3.0. The format includes relational metadata for episode‑level queries and offers native windowing and batching for robot learning workloads.
TL;DR Today we release LeRobotDataset:v3 ! In our previous LeRobotDataset:v2 release, we stored one episode per file, hitting file-system limitations when scaling datasets to millions of episodes. LeRobotDataset:v3 packs multiple episodes in a single file, using relational metadata to retrieve information at the individual episode level from multi-episode files. The new format also natively supports accessing datasets in streaming mode, allowing to process large datasets on the fly.We provide a one-liner util to convert all datasets in the LeRobotDataset format to the new format, and are very excited to share this milestone with the community ahead of our next stable release!
LeRobotDataset is a standardized dataset format designed to address the specific needs of robot learning, and it provides unified and convenient access to robotics data across modalities, including sensorimotor readings, multiple camera feeds and teleoperation status. Our dataset format also stores general information regarding the way the data is being collected ( metadata ), including a textual description of the task being performed, the kind of robot used and measurement details like the frames per second at which both image and robot state streams are sampled. Metadata are useful to index and search across robotics datasets on the Hugging Face Hub!
Within lerobot , the robotics library we are developing at Hugging Face, LeRobotDataset provides a unified interface for working with multi-modal, time-series data, and it seamlessly integrates both with the Hugging Face and Pytorch ecosystems. The dataset format is designed to be easily extensible and customizable, and already supports openly available datasets from a wide range of embodiments—including manipulator platforms such as the SO-100 arms and ALOHA-2 setup, real-world humanoid data, simulation datasets, and even self-driving car data! You can explore the current datasets contributed by the community using the dataset visualizer ! 🔗
Most robot learning algorithms, based on reinforcement learning (RL) or behavioral cloning (BC), tend to operate on a stack of observations and actions. For instance, RL algorithms typically use a history of previous observations o_{t-H_o:t} , and BC algorithms are instead typically trained to regress chunks of multiple actions. To accommodate for the specifics of robot learning training, LeRobotDataset provides a native windowing operation, whereby we can use the seconds before and after any given observation using a delta_timestamps argument.
lerobot is the end-to-end robotics library developed at Hugging Face, supporting real-world robotics as well as state of the art robot learning algorithms. The library allows to record datasets locally directly on real-world robots, and to store datasets on the Hugging Face Hub. You can read more about the robots we currently support here !
Besides scale, this new release of LeRobotDataset also enables support for a streaming functionality, allowing to process batches of data from large datasets on the fly, without having to download prohibitively large collections of data onto disk. You can access and use any dataset in v3.0 in streaming mode by using the dedicated StreamingLeRobotDataset interface! Streaming datasets is a key milestone towards more accessible robot learning, and we are excited about sharing it with the community 🤗
LeRobotDataset:v3 is going to be a part of the lerobot library starting from lerobot-v0.4.0 , and we are very excited about sharing it early with the community. You can install the latest lerobot-v0.3.x supporting this new dataset format directly from PyPI using:
Once you have installed a version of lerobot which supports the new dataset format, you can record a dataset with our signature robot arm, the SO-101, by using teleoperation alongside the following instructions:
Head to the official documentation to see how to record a dataset for your use case.
LeRobotDataset:v3.0 will be released with lerobot-v0.4.0 , together with the possibility to easily convert any dataset currently hosted on the Hugging Face Hub to the new v3.0 using:
Note that this is a pre-release, and generally unstable version. You can follow the status of the development of our next stable release here !
The new format enables large‑scale robot learning by allowing researchers to handle millions of episodes without local storage constraints, accelerating data‑driven advances in robotics.