TL;DR: Today, Meta releases Llama Guard 4, a 12B dense (not a MoE!) multimodal safety model, and two new Llama Prompt Guard 2 models. This release comes with multiple open model checkpoints, along with an interactive notebook for you to get started easily 🤗. Model checkpoints can be found in Llama 4 Collection .
Vision and large language models deployed to production can be exploited to generate unsafe output through jail breaking image and text prompts. Unsafe content in production varies from being harmful or inappropriate to violating privacy or intellectual property.
New safeguard models address this issue by evaluating image and text, and the content generated by the model. User messages classified as unsafe are not passed to vision and large language models, and unsafe assistant responses can be filtered out by production services.
Llama Guard 4 is a new multimodal model designed to detect inappropriate content in images and text, whether used as input or generated as output by the model. It’s a dense 12B model pruned from Llama 4 Scout model, and it can run on a single GPU (24 GB of VRAM). It can evaluate both text-only and image+text inputs, making it suitable for filtering both inputs and outputs of large language models. This enables flexible moderation pipelines where prompts are analyzed before reaching the model, and generated responses are reviewed afterwards for safety. It can also understand multiple languages.
The model can classify 14 types of hazard defined in the MLCommons hazard taxonomy, along with code interpreter abuse.
The list of categories detected by the model can be configured by the user on inference, as we’ll see later.
Llama Guard 4 employs a dense feedforward early-fusion architecture, in contrast to Llama 4 Scout, which uses Mixture-of-Experts (MoE) layers with one shared dense expert and sixteen routed experts per layer. To leverage Llama 4 Scout pre-training, the architecture is pruned into a dense model by removing all routed experts and router layers, retaining only the shared expert. This results in a dense feedforward model initialized from the pre-trained shared expert weights. No additional pre-training is applied to Llama Guard 4. The post-training data consists of multi-image training data up to 5 images and human-annotated multilingual data, previously used to train Llama Guard 3 models. The training data consists of 3:1 text-only to multimodal data.
Below you can find the performance of Llama Guard 4 compared against Llama Guard 3, the previous iteration of the safety model.
The Llama Prompt Guard 2 series introduces two new classifiers with 86M and 22M parameters, focused on detecting prompt injections and jailbreaks. Compared to its predecessor, Llama Prompt Guard 1, this new version offers improved performance, a faster and more compact 22M model, tokenization resistant to adversarial attacks, and simplified binary classification (benign vs. malicious).
To use Llama Guard 4 and Prompt Guard 2, make sure you have hf_xet and the preview release of transformers for Llama Guard installed.
Here is a simple snippet of how to run Llama Guard 4 on the user inputs.
If your application does not require moderation on some of the supported categories, you can ignore the ones you are not interested in, as follows:
Sometimes it is not just the user input, but also the model’s generations that can contain harmful content. We can also moderate the model’s generation!
This works because the chat template generates a system prompt that does not mention the excluded categories as part of the list of categories to watch for.
You can use Llama Prompt Guard 2 directly via the pipeline API:
Alternatively, it can also be used via AutoTokenizer + AutoModel API: