Opening article
Google has released PaliGemma 2 mix, a fine‑tuned version of its PaliGemma vision‑language models. The mix models are trained on a variety of vision‑language tasks such as OCR, long and short captioning, and more, and are available in multiple sizes (3B, 10B, 28B) and resolutions (224x224, 448x448, 896x896). They are intended to provide a quick performance benchmark for downstream fine‑tuning and to demonstrate how pre‑trained checkpoints perform on mixed academic datasets.
Last December, Google released PaliGemma 2: a new family of pre-trained ( pt ) PaliGemma vision language models (VLMs) based on SigLIP and Gemma 2 . The models come in three different sizes (3B, 10B, 28B) and three different resolutions (224x224, 448x448, 896x896).
Today, Google is releasing PaliGemma 2 mix : fine-tuned on a mix of vision language tasks, including OCR, long and short captioning and more.
PaliGemma 2 pretrained ( pt ) variants are great vision language models to transfer on a given task at hand. All pt checkpoints are meant to be fine-tuned on a downstream task and were released for that purpose.
The mix models give a quick idea of the performance one would get when fine-tuning the pre-trained checkpoints on a downstream task. The main purpose of the PaliGemma model family is to provide pretrained models that can learn better on a downstream task, instead of providing a versatile chat model. Mix models give a good signal of how pt models perform when fine-tuned on a mix of academic datasets.
PaliGemma 2 mix models can accomplish a variety of tasks. We can categorize them according to their subtasks as follows.
Note that this list of subtasks is non-exhaustive, and you can get more information on the full list of tasks in the PaliGemma 2 paper .
The release offers a ready‑to‑use benchmark for evaluating vision‑language models on diverse tasks, aiding researchers and developers in selecting and fine‑tuning models for specific applications.