Managing risks from increasingly capable open-weight AI systems

This summer, several powerful open-weight AI systems were released. All are powerful reasoning models – and all have their parameters openly available for download on the internet. Reported evaluation results suggest that they are the most capable open-weight AI systems to date, knocking on the door of even today’s most advanced closed-weight systems.

Open-weight systems are the lifeblood of open research and innovation in AI. They increase transparency, allow for widespread red teaming, and decrease market concentration. However, open-weight models also pose unique risks. They can allow harmful AI capabilities to proliferate rapidly and irreversibly. Just as they can be modified for beneficial uses, they can also be modified for harmful ones. Safeguards designed to prevent harmful behaviours can also be quickly and cheaply removed from open-weight models.

In this post, we discuss existing methods and open problems for open-weight model risk management – including new research from the UK AI Security Institute in collaboration with EleutherAI and the University of Oxford.

If a model is deployed with closed weights, its deployer can draw from a rich toolkit for making it safer. For example, they can scaffold the model with safeguards such as content filters. They also fully control the points of access, allowing them to enforce acceptable usage policies.

Model provenance: A key goal for monitoring the open-weight model ecosystem is to have techniques for identifying and tracing models in the wild. These include methods for studying model ‘heritage’ and ‘watermarking’ the weights of models. These techniques can be circumvented with some effort, but they could nonetheless be useful, much like fingerprinting in criminal forensics. Future work on benchmarking and refining model forensics techniques will be useful, but the main barrier to adoption is likely to be implementation and integration with model distribution platforms.

Pulling and replacing systems: Once a system is released with open weights, it cannot be rolled back. However, halting download access can still slow proliferation. Developers can also work to accelerate the obsolescence of a harmful system by replacing a pulled system with a safer version.

Not deploying high-risk systems with open weights: Avoiding open-weight release of a system can be a strong defence against many downstream risks that they pose.

Staged deployment: AI model deployment is not a binary process. There is a continuum of deployment options ranging from fully closed to fully open. Rather than openly releasing a system all at once, AI developers can choose to release it in stages and monitor its uses and impacts at each stage before a final full release.

Open-weight models, on the other hand, are harder to safeguard, because they can be shared and modified arbitrarily without oversight. This makes developing strong safety assurances harder. Risk management for open-weight general-purpose AI systems will likely benefit from leveraging a multi-pronged strategy to minimise harms.

While no current solutions can offer hard guarantees of safety, various strategies can meaningfully reduce risks, especially in combination. In the table below, we summarise techniques for monitoring and mitigating risks from open-weight models.

Techniques vary in their effectiveness and robustness, as well as the extent to which more research is needed to understand them. The ratings below represent best guesses of technical researchers on the Safeguards team at AISI.