Welcoming Llama Guard 4 on Hugging Face Hub

TL;DR: Today, Meta releases Llama Guard 4, a 12B dense (not a MoE!) multimodal safety model, and two new Llama Prompt Guard 2 models. This release comes with multiple open model checkpoints, along with an interactive notebook for you to get started easily 🤗. Model checkpoints can be found in Llama 4 Collection .

Vision and large language models deployed to production can be exploited to generate unsafe output through jail breaking image and text prompts. Unsafe content in production varies from being harmful or inappropriate to violating privacy or intellectual property.

New safeguard models address this issue by evaluating image and text, and the content generated by the model. User messages classified as unsafe are not passed to vision and large language models, and unsafe assistant responses can be filtered out by production services.

The Llama Prompt Guard 2 series introduces two new classifiers with 86M and 22M parameters, focused on detecting prompt injections and jailbreaks. Compared to its predecessor, Llama Prompt Guard 1, this new version offers improved performance, a faster and more compact 22M model, tokenization resistant to adversarial attacks, and simplified binary classification (benign vs. malicious).

To use Llama Guard 4 and Prompt Guard 2, make sure you have hf_xet and the preview release of transformers for Llama Guard installed.

Llama Guard 4 employs a dense feedforward early-fusion architecture, in contrast to Llama 4 Scout, which uses Mixture-of-Experts (MoE) layers with one shared dense expert and sixteen routed experts per layer. To leverage Llama 4 Scout pre-training, the architecture is pruned into a dense model by removing all routed experts and router layers, retaining only the shared expert. This results in a dense feedforward model initialized from the pre-trained shared expert weights. No additional pre-training is applied to Llama Guard 4. The post-training data consists of multi-image training data up to 5 images and human-annotated multilingual data, previously used to train Llama Guard 3 models. The training data consists of 3:1 text-only to multimodal data.

Llama Guard 4 is a new multimodal model designed to detect inappropriate content in images and text, whether used as input or generated as output by the model. It’s a dense 12B model pruned from Llama 4 Scout model, and it can run on a single GPU (24 GB of VRAM). It can evaluate both text-only and image+text inputs, making it suitable for filtering both inputs and outputs of large language models. This enables flexible moderation pipelines where prompts are analyzed before reaching the model, and generated responses are reviewed afterwards for safety. It can also understand multiple languages.

The model can classify 14 types of hazard defined in the MLCommons hazard taxonomy, along with code interpreter abuse.

The list of categories detected by the model can be configured by the user on inference, as we’ll see later.