Open R1: How to use OlympicCoder locally for coding

Everyone’s been using Claude and OpenAI as coding assistants for the last few years, but there’s less appeal if you look at the developments coming out of open source projects like Open R1 . If we look at the evaluation on LiveCodeBench below, we can see that the 7B parameter variant outperforms Claude 3.7 Sonnet and GPT-4o. These models are the daily drivers of many engineers in applications like Cursor and VSCode.

Evals are great and all, but I want to get my hands dirty and feel the commits! This blog post focuses on how you can integrate these models in your IDE now. We will set up OlympicCoder 7B, the smaller of the two OlympicCoder variants, and we’ll use a quantized variant for optimum local inference. Here’s the stack we’re going to use:

It’s important to say that we chose this stack purely for simplicity. You might want to experiment with the larger model and/or different GGUF files. Or even alternative inference engines like llama.cpp .

LM Studio is like a control panel for AI models. It integrates with the Hugging Face hub to pull models, helps you find the right GGUF file, and exposes an API that other applications can use to interact with the model.

The GGUF files that we need are hosted on the hub. We can open the model from the hub in LMStudio, using the ‘Use this model’ button:

This will link to the LMStudio application and open it on your machine. You’ll just need to Choose a Quantization. I went for Q4_K_M because it will perform well on most devices. If you have more compute, you might want to try out one of the options with Q8_* .