Opening article
Today Google releases a new and better family of multilingual vision-language encoders, SigLIP 2 . The authors have extended the training objective of SigLIP ( sigmoid loss ) with additional objectives for improved semantic understanding, localization, and dense features. SigLIP 2 models outperform the older SigLIP ones at all model scales in core capabilities, including zero-shot classification, image-text retrieval, and transfer performance when extracting visual representations for Vision-Language Models (VLMs). SigLIP 2 is just that, a bunch of really interesting and smart training objectives applied on top of that of SigLIP's to provide better and stronger vision language encoders.
Today Google releases a new and better family of multilingual vision-language encoders, SigLIP 2 . The authors have extended the training objective of SigLIP ( sigmoid loss ) with additional objectives for improved semantic understanding, localization, and dense features.
SigLIP 2 models outperform the older SigLIP ones at all model scales in core capabilities, including zero-shot classification, image-text retrieval, and transfer performance when extracting visual representations for Vision-Language Models (VLMs).
A cherry on top is the dynamic resolution ( naflex ) variant. This is useful for downstream tasks sensitive to aspect ratio and resolution.
To improve fine-grained local semantics in image representation, we introduce two key training objectives, Global-Local Loss, and Masked Prediction Loss. Taking inspiration from self-supervised learning literature, we use self-distillation . We can use a model as a teacher, and the same model as a student. Upon each iteration the teacher will be the moving average of the student's parameters.
In no particular order we would like to thank Pavel , Ross , Pablo , Pedro , Lysandre and the rest of the Hugging Face team for their immense support and contribution towards this project.
Vision encoders are simple - they take an image, encode it into a representation, and that representation is used for downstream tasks like classification, object detection, image segmentation, and more vision tasks. Researchers are always in pursuit of visual representations that are dense , locality-aware , and semantically rich .
CLIP and ALIGN are the first examples of image encoders and text encoders aligned together through joint training. This approach opened new ways to train vision models. SigLIP took it further, replacing CLIP's contrastive loss with sigmoid loss for even better encoders.
The takeaway? With smarter training objectives, we keep building vision encoders that are more structured, fine-grained, and powerful. SigLIP 2 is just that, a bunch of really interesting and smart training objectives applied on top of that of SigLIP's to provide better and stronger vision language encoders.
SigLIP 2: A better multilingual vision language encoder. Covered by Hugging Face on February 21, 2025.