Run AI workloads on any cloud, store on Hugging Face: zero-egress storage with SkyPilot

Hugging Face and SkyPilot have partnered to introduce zero-egress storage for artificial intelligence workloads. According to the announcement, models and datasets can remain on the Hugging Face Hub while SkyPilot schedules compute tasks on whichever cloud cluster has available GPUs. Users can mount a Hugging Face bucket or repository into a SkyPilot job using an hf:// URL scheme and an existing HF_TOKEN. Because Hugging Face charges no egress fees, reading data onto GPUs is free across any cloud environment. The storage integration utilizes the hf-mount FUSE backend, enabling lazy reads where code can process data before the entire file is downloaded, thus keeping expensive GPUs busy immediately. The hf:// scheme covers the complete workflow, including reading models, writing checkpoints during training, publishing finished models, and pulling them for inference servers. Hugging Face buckets are built on Xet technology, which provides automatic deduplication using content-defined chunking. Authentication relies on a single token that functions across platforms such as AWS, GCP, Azure, Nebius, Lambda, and on-premises Kubernetes clusters. The collaboration aims to solve the challenge of regional object storage silos, allowing teams to utilize reserved GPU capacity across multiple vendors without paying hefty cross-cloud transfer taxes or maintaining duplicate data copies.