PRX Part 4: Our Data Strategy

In Part 4 of the PRX series, Hugging Face details the data strategy used to build the pre-training pipeline for the PRX model. The team assembled a large, diverse dataset combining public and internal sources, prioritizing breadth and diversity over per-image perfection. To process the data effectively, Hugging Face utilized long, faithful captions generated by a Vision-Language Model to describe all image elements, turning potential noise into controllable attributes. For dataset infrastructure, the pipeline uses Lance for feature engineering and dataset curation due to its column format and vector search capabilities, paired with Mosaic Streaming and Mosaic MDS for distributed training and object storage compatibility. After switching the text encoder to Qwen3-VL, the team computed text latents on the fly during training rather than pre-computing them, incurring only a 3 to 4 percent throughput cost while keeping MDS shards small enough to store on a local SLURM cluster filesystem. Additionally, Hugging Face opted to encode all images as JPEGs at quality 92 rather than using lossless PNGs. Through testing, they found that high-quality JPEG storage was imperceptible compared to PNGs and had a minimal effect on output model quality while significantly reducing storage requirements. For data ingest, the team relies on Ray Data to read source tables in parallel and write fragments across a Lance table cluster.