Granite 4.1 is a family of dense, decoder‑only language models (3B, 8B, 30B) trained from scratch on approximately 15 trillion tokens. The training pipeline is divided into five distinct phases: the first two focus on foundational pre‑training with a broad web‑scale data mix, the next two perform mid‑training with progressively higher‑quality, domain‑specific data and a shift toward reasoning and instruction data, and the final phase extends the context window to 512K tokens through a staged long‑context extension process. Each phase uses a different data mixture and learning‑rate schedule, gradually moving from web‑heavy to quality‑heavy content. After pre‑training, the models undergo supervised fine‑tuning on about 4.1 million curated samples. This fine‑tuning employs an LLM‑as‑Judge framework that evaluates assistant responses across six weighted dimensions—instruction following, correctness, completeness, conciseness, naturalness, and calibration—alongside a deterministic rule‑based pipeline that enforces structural integrity and deduplication. The resulting high‑quality dataset is then used to fine‑tune the base models. Finally, a multi‑stage reinforcement learning pipeline further refines the models. This pipeline includes multi‑domain RL, RLHF, identity and knowledge‑calibration RL, and math RL, using on‑policy GRPO with DAPO loss (though DAPO is disabled during training for computational efficiency). The 8B instruct model matches or surpasses the previous Granite 4.0‑H‑Small (32B‑A9B MoE) despite its simpler dense architecture and fewer parameters. All Granite 4.1 models are released under the Apache 2.0 license.
Granite 4.1 LLMs: How They’re Built
Granite 4.1 is a family of dense, decoder‑only language models (3B, 8B, 30B) trained from scratch on about 15 trillion tokens. The training pipeline consists of five stages: foundational pre‑training, mid‑training with higher‑quality data, and a final long‑context extension to 512K tokens. After pre‑training, the models undergo supervised fine‑tuning on roughly 4.1 million curated samples using an LLM‑as‑Judge framework, followed by a multi‑stage reinforcement learning process that includes multi‑domain RL, RLHF, identity and knowledge‑calibration RL, and math RL.
Why it matters
The Granite 4.1 approach demonstrates that careful data curation and staged training can produce competitive instruction‑following models with fewer parameters, informing efficient model design in the industry.