Early lessons from evaluating frontier AI systems

Note to readers: we changed our name to the AI Security Institute on 14 February 2025. Read more here.

The AI Safety Institute's (AISI) mission is to equip governments with an empirical understanding of the safety of frontier AI systems. To achieve this, we design and run evaluations that aim to measure the capabilities of AI systems that may pose risks.

Since November 2023, we have built one of the largest safety evaluation teams globally with a leadership team of world class researchers, established a Memorandum of Understanding (MoU) and begun to operationalise joint testing with the US AISI. We have also published an open-source framework for evaluations , reported our results of testing 4 large language models from major companies, and conducted several pre-deployment tests.

But this is just the beginning. Model evaluations are a new and rapidly evolving science, and we routinely iterate on our approach. We will continue to grow our team and build our capacity to address challenges and open scientific questions. If you are excited by this, please join us !

This blog sets out our thinking to date on how to design and run third-party evaluations, including key elements to consider and open questions. This is not intended to provide robust recommendations; rather we want to start a conversation in the open about these practices and to learn from others.

We discuss the role of third-party evaluators and what they could target for testing, including which systems to test, when to test them, and which risks and capabilities to test for. We also examine how to evaluate effectively, including which tests to use for which purpose, how to develop robust testing, and how to ensure safety and security.