Multilingual Bridges: How Data Mixing Unlocks In-Language Reasoning

Some of the biggest recent gains in language models have come from a simple idea: let the model ‘think’ before it answers. The result of this idea is the development of reasoning models that work through a problem step by step, producing a written ‘reasoning trace’. This has led to great improvements in language models’ abilities to complete tasks like math and coding. These models, however, mainly focus on giving the right answer, but we ask an often-overlooked question: can users actually follow the reasoning traces?

For many users, the answer today is no: when a question is asked in Spanish, Arabic, or Swahili, these models tend to reason in English. The final answer may come back in the user's language, but the reasoning steps behind that answer stay out of reach for anyone who doesn't speak English. This is not an unsolvable problem. By enabling multilingual reasoning models to reason in the user’s language, which we call ‘ in-language ’ or ‘ L2 ’ reasoning, users can read the reasoning trace, verify each step, find where things went wrong when the answer is incorrect, and step in when needed.

L2 reasoning may also help the model itself. Much knowledge, such as cultural context and linguistic nuance, is expressed most naturally in its source language. L2 reasoning could let models draw on this culturally embedded knowledge more effectively, especially in linguistic and open-ended generation tasks. The example below shows what this looks like in practice: Asking in Hindi which event signals good luck, our Tiny Aya L2-Thinker model reasons in Hindi about what Indian superstitions hold and reaches the correct answer, while thinking in English asserts a Western-biased answer without weighing the options.

The exception is PolyMath, a competition-level math benchmark with problems well beyond grade-school arithmetic. Here accuracy falls more noticeably. We suspect this is because our model never went through a reinforcement learning stage, which hard math benefits from most and which could also smooth over artifacts introduced when we translated our training traces; this is a future direction that could be pursued with a focus on mathematics.

Tiny Aya L2-Thinker is the only model that stays strong on all three axes as resource availability declines. Its L2 reasoning rate sits near the ceiling on tiers 1 and 2 (99%) and drops only slightly on the lowest two (94.3% and 94.5%). In contrast, Magistral collapses from 72% to 5% L2 reasoning rate, M-Thinker degrades on every axis and its traces roughly double in length, and language-forced Qwen keeps its L2 reasoning rate fairly high, but at the cost of accuracy and efficiency.

To support further research on accessible, in-language reasoning, we release our model and multilingual reasoning data on Hugging Face.

In this blog post, we describe our recent research that shows how better data mixing leads to effective L2 reasoning without hurting accuracy much. Building on the Tiny Aya base model with 32K context length , we apply our data mixing strategy to create Tiny Aya L2-Thinker , a massively multilingual model achieving over 93% in-language reasoning rate across 60 languages.

Getting models to reason in the user's language is not a new idea, but earlier attempts have fallen short in three ways. First, they show that L2 reasoning often comes at a cost: models that are pushed to reason in another language tend to get more answers wrong , and approaches that avoid this drop rely on complex, expensive training . Second, most studies evaluate models only on math and science problems. Math is a good test of reasoning, but it's also the task where language matters least, so it tells us little about cultural knowledge or open-ended writing. Third, they cover only a handful of languages, usually ones that already receive a large amount of attention from AI researchers and developers.

Our model, Tiny Aya L2-Thinker, addresses all three of these shortcomings: it reasons in the user's language while staying nearly as accurate as reasoning in English across a range of tasks, with the exception of competition-level math. We test it on math reasoning, commonsense understanding, cultural reasoning, open-ended writing, and instruction following. It also supports 45 languages in training, with evidence that it can carry this ability over to languages it was never shown reasoning examples in.