Strengthening our defenses against distillation attacks

In recent months, we’ve made substantial progress on reducing the effectiveness of distillation attacks on Claude using new methods, as we outline in this blog. Early evidence suggests that these methods have reduced the amount of data harvested by distillers by around 90 to 99%.

Distillation attacks are covert, industrial-scale campaigns to extract the capabilities of one model and replicate them in another, without the authorization of the original model’s developers. They are often conducted using networks of fraudulent accounts. These attacks allow companies to mimic frontier capabilities for a fraction of the development time and cost. As we wrote in September , this method has been used extensively to help overseas AI laboratories keep pace with US companies.

We’re concerned about illicit distillation for two reasons. First, distillation enables companies to build models that approach the capability frontier, but which have, in practice, been released without proportionally robust safeguards. This means that distilled models could be used to cause significant harm in areas like biology and cybersecurity .

Zhipu, for example , distilled the cyber capabilities of leading US models during the development of GLM-5.3. The company tried to distill Claude Fable 5—although, after finding that our cyber safeguards degraded its attacks, it switched to an older Claude model and another lab’s model. The US Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) has since assessed GLM-5.3 to be the most cyber-capable open-weights model released to date. Our own research has found that GLM-5.3’s safeguards can easily be bypassed, and that the model is willing to develop security exploits from end-to-end.

Second, illicitly distilled models present a national security risk. The companies that distill US models are integrating the capabilities they gain from distillation into foreign military, intelligence, and surveillance systems . For example, we recently reported on a case in which requests to Moonshot’s Kimi models were secretly forwarded to Claude as part of its distillation efforts. One request came from a user (who we believe was affiliated with China’s People’s Liberation Army) who sought to analyze CCTV surveillance of a specific individual. These are the kinds of capabilities distillers are hoping to develop.

In response to the increasing sophistication and scale of these attacks, we’ve expanded and greatly strengthened our defenses against them. As we’ll describe below, these measures have been successful—although they do involve trade-offs for both Anthropic and our customers. We’ve turned away meaningful revenue and trillions of tokens’ worth of paying traffic from distillers, and our research and product teams have invested substantial time and effort into this problem. Some of our changes have also required legitimate users to make updates to their integrations.

Distillation is a problem that every model developer faces and must confront. We hope this post helps developers and platforms better understand how they can prevent these attacks, what the trade-offs with these measures might be, and why the costs are worth bearing.

Distillers aim to generate millions of responses from a larger, more capable model (in this case, Claude), then use those responses to teach a smaller model to mimic the more capable model’s behavior. The most valuable kind of data for this purpose is transcripts of Claude’s reasoning—the thinking process that Claude writes out before it responds. This is much more useful than its output alone, since it helps the smaller model understand how Claude reaches its final answer. Our defenses against distillation therefore focus almost entirely on making Claude’s reasoning harder for distillers to access.