How to deploy and fine-tune DeepSeek models on AWS

A running document to showcase how to deploy and fine-tune DeepSeek R1 models with Hugging Face on AWS.

If you’ve ever struggled with a tough math problem, you know how useful it is to think a little longer and work through it carefully. OpenAI’s o1 model showed that when LLMs are trained to do the same—by using more compute during inference—they get significantly better at solving reasoning tasks like mathematics, coding, and logic.

However, the recipe behind OpenAI’s reasoning models has been a well kept secret. That is, until last week, when DeepSeek released their DeepSeek-R1 model and promptly broke the internet (and the stock market! ).

You can deploy the Deepseek distilled models on Amazon Bedrock via the marketplace, which will deploy an endpoint in Amazon SageMaker AI under the hood. Here is a video of how you can navigate through the AWS console:

Because you are using a TGI v3 container under the hood, the most performant parameters for the given hardware will be automatically selected.

DeepSeek AI open-sourced DeepSeek-R1-Zero, DeepSeek-R1, and six dense models distilled from DeepSeek-R1 based on Llama and Qwen architectures. You can find them all in the DeepSeek R1 collection .

We collaborate with Amazon Web Services to make it easier for developers to deploy the latest Hugging Face models on AWS services to build better generative AI applications.

Let’s review how you can deploy and fine-tune DeepSeek R1 models with Hugging Face on AWS.