Opening article
Open‑R1 is a community effort to replicate DeepSeek R1’s training pipeline and synthetic data. Within a week the team confirmed that they can reproduce DeepSeek’s MATH‑500 benchmark results and have begun scaling generation to support dataset creation. The project has released an evaluation leaderboard, integrated GRPO into TRL 0.14, and explored vLLM configurations to handle the model’s long outputs. Media coverage from major outlets has highlighted the initiative and the broader interest in DeepSeek R1’s performance and training cost.
It’s been two weeks since the release of DeepSeek R1 and just a week since we started the open-r1 project to replicate the missing pieces, namely the training pipeline and the synthetic data. This post summarizes:
It should serve both as an update on the project and as a collection of interesting resources around DeepSeek-R1.
Let’s start by looking at the progress we made on Open-R1. We started Open-R1 just one week ago and people across the teams as well as the community came together to work on it and we have some progress to report.
The first step in reproduction is to verify that we can match the evaluation scores. We are able to reproduce Deepseek's reported results on the MATH-500 Benchmark:
You can find the instructions to run these evaluations in the open-r1 repository .
One observation we have made is the enormous size of the generations from the DeepSeek models, which makes even evaluating the model challenging. Here we show DeepSeek-R1 response lengths in the OpenThoughts dataset:
Reproducing a state‑of‑the‑art LLM’s training pipeline enables broader research and potential cost‑efficient alternatives. The effort also demonstrates how community collaboration can accelerate model understanding and dataset creation.