Google DeepMind’s Gemma 4 multimodal family has been released on Hugging Face, providing open‑source models under an Apache 2 license that handle text, image, audio, and video inputs. The family includes five sizes, all base and instruction fine‑tuned, and is designed for compatibility with popular inference engines such as transformers, llama.cpp, MLX, WebGPU, and Rust. Gemma 4 introduces Per‑Layer Embeddings, which give each transformer layer a small dedicated vector to modulate hidden states, and a shared KV cache that reuses key and value tensors in the last layers to reduce compute and memory usage. The 12B Unified variant eliminates separate vision or audio encoders, projecting raw image patches and audio waveforms directly into the LLM embedding space, thereby reducing multimodal latency and enabling single‑pass fine‑tuning. Benchmarks show the 31B dense model achieving an estimated LMArena score of 1452 and the 26B Mixture‑of‑Experts model reaching 1441 with only 4B active parameters. The models perform well on tasks such as OCR, speech‑to‑text, object detection, captioning, and function calling, and can output JSON for bounding boxes without special prompts. Gemma 4 is available in ONNX checkpoints for edge and browser deployment and supports first‑class integration with the transformers library, bitsandbytes, PEFT, and TRL. The release highlights the feasibility of deploying large multimodal models on a wide range of devices, making advanced AI capabilities more accessible to developers and end users.
Welcome Gemma 4: Frontier multimodal intelligence on device
Google DeepMind’s Gemma 4 multimodal family has been released on Hugging Face, offering open‑source models under Apache 2 that support text, image, audio, and video inputs. The models come in five sizes, all base and instruction fine‑tuned, and are compatible with popular inference engines such as transformers, llama.cpp, MLX, WebGPU, and Rust. Gemma 4 introduces Per‑Layer Embeddings and a shared KV cache to improve efficiency and multimodal performance, while the 12B Unified variant eliminates separate vision or audio encoders for lower latency and easier deployment on consumer hardware. Benchmarks show the 31B dense model achieving an estimated LMArena score of 1452 and the 26B MoE reaching 1441 with only 4B active parameters.
Why it matters
The release demonstrates that large multimodal models can be made truly open, efficient, and deployable on a wide range of hardware, lowering barriers for developers to integrate advanced AI into edge and browser applications.