Opening article
Gemma 3n, a multimodal model announced at Google I/O, is now fully available on major open‑source libraries such as transformers, timm, MLX, llama.cpp, transformers.js, and ollama. Two sizes—gemma‑3n‑E2B and gemma‑3n‑E4B—offer 5 B and 8 B parameters but require only 2 B and 4 B of VRAM, with the E2B variant running on as little as 2 GB GPU RAM. The release includes ONNX weights, a Hugging Face Space for demos, and notebooks for fine‑tuning across modalities.
Gemma 3n was announced as a preview during Google I/O. The on-device community got really excited, because this is a model designed from the ground up to run locally on your hardware. On top of that, it’s natively multimodal , supporting image, text, audio, and video inputs 🤯
Today, Gemma 3n is finally available on the most used open source libraries. This includes transformers & timm, MLX, llama.cpp (text inputs), transformers.js, ollama, Google AI Edge, and others.
This post quickly goes through practical snippets to demonstrate how to use the model with these libraries, and how easy it is to fine-tune it for other domains.
Two model sizes have been released today, with two variants (base and instruct) each. The model names follow a non-standard nomenclature: they are called gemma-3n-E2B and gemma-3n-E4B . The E preceding the parameter count stands for Effective . Their actual parameter counts are 5B and 8B , respectively, but thanks to improvements in memory efficiency, they manage to only need 2B and 4B in VRAM (GPU memory).
Finally, we are also releasing ONNX weights for the gemma-3n-E2B-it model variant, enabling flexible deployment across diverse runtimes and platforms. For JavaScript developers, Gemma3n has been integrated into Transformers.js and is available as of version 3.6.0 .
These models, therefore, behave like 2B and 4B in terms of hardware support, but they punch over 2B/4B in terms of quality. The E2B model can run in as little as 2GB of GPU RAM, while E4B can run with just 3GB of GPU RAM.
Support for videos is done as a collection of frames of images
Gemma 3n comes with day 0 support for MLX across all 3 modalities. Make sure to upgrade your mlx-vlm installation.
In addition to the language decoder, Gemma 3n uses an audio encoder and a vision encoder . We highlight their main features below, and describe how they have been added to transformers and timm , as they are the reference for other implementations.
The easiest way to vibe check the model is with the dedicated Hugging Face Space for the model. You can try out different prompts, with different modalities, here.
Install the latest version of timm (for the vision encoder) and transformers to run inference, or if you want to fine tune it.
The release lowers hardware barriers for multimodal AI, enabling broader experimentation and deployment across diverse platforms, which can accelerate innovation in AI applications.