Opening article
You've got everything ready - data, model, a beefy GPU setup. After some detective work on our nanoVLM project, we discovered the real culprit wasn't our model or hardware, it was our data pipeline being incredibly wasteful. To make it easier to follow the data preparation tasks, we created a separate repo laser-focused on the data pipeline only. Padding is the enemy, and we need a strategy to minimize it while maximizing how much data we can fit into each batch.
You've got everything ready - data, model, a beefy GPU setup. You hit "run" and... wait. And wait some more. Your GPUs are barely breaking a sweat while your wallet's getting lighter by the hour.
Sound familiar? We've been there. After some detective work on our nanoVLM project, we discovered the real culprit wasn't our model or hardware, it was our data pipeline being incredibly wasteful.
In this post we build an efficient pipeline in five stages . In each stage we add or remove from the previous step and comment on what went right and what did not.
To make it easier to follow the data preparation tasks, we created a separate repo laser-focused on the data pipeline only. We hope this will be much easier to understand that reading the code once integrated with the nanoVLM repository. In addition, this could be useful to bootstrap other data pipelines!
To follow along, all you need to do is clone the repository. It contains the final data preparation tasks, but it's designed to showcase each step of the way.
Before optimizing anything, we need to understand what we are working with. Our multimodal dataset has images, text prompts, and responses.
Efficient MultiModal Data Pipeline. Covered by Hugging Face on July 8, 2025.