Now in Nature: Retrofitting language models to operate over bytes

Allen AI announced that the research behind its Bolmo byte‑level language models has been published in Nature and that it is releasing new checkpoints on Hugging Face. The paper describes a technique called byteifying, which takes an existing subword model and converts it into a byte‑level model with a relatively short additional training run. Using this method, Allen AI produced Bolmo 1B and Bolmo 7B from its open Olmo models and demonstrated that these models are competitive with strong subword models across a broad range of tasks. The Nature paper also shows that byteifying generalizes beyond Olmo; the team applied the same process to Qwen 3 8B and Llama 3 8B, creating the byte‑level models Bwen 8B and Blama 8B. Both new models come close to matching the performance of their source models, with Bwen 8B outperforming Bolmo 7B in the aggregate evaluation suite. Allen AI is also releasing Stage 1 checkpoints that keep the original global model frozen while training the new byte‑level components, providing researchers with a faster starting point for experimentation. Developers have already begun fine‑tuning Bolmo for specific applications, such as a Russian‑to‑English poetry and lyric translation model. The publication follows other recent Allen AI research in Nature, including a February paper on retrieval‑augmented language models for scientific literature synthesis. By making Bolmo and its training recipe fully open, Allen AI aims to enable the research community to inspect, reproduce, and build upon these advances, potentially leading to systems that adapt their representations across languages, domains, and tasks and that can extend beyond text to other data modalities.