Early Insights from Developing Question-Answer Evaluations for Frontier AI

Note to readers: we changed our name to the AI Security Institute on 14 February 2025. Read more here.

A common method 1 for quickly assessing an AI model’s capabilities involves prompting models to answer hundreds of questions, before automatically scoring the answers. If the questions are multiple-choice then automating the scoring is easy. But we can get a much more complete understanding of a model’s capabilities if we also ask open-ended questions, for which many possible answers are valid. The challenge lies in writing and automatically grading these open-ended questions whilst ensuring the results are as informative as possible.

At the AI Safety Institute, we’ve been working on open-ended question and answer testing for several months. This blog post shares our experiences and key insights from this work. We hope that sharing this will help advance the science of evaluations, and provide some practical suggestions for others creating their own evaluations.

Considering a range of use cases for LLMs makes capabilities elicitation difficult. Ideally, we would perform capability elicitation multiple times, each time only using tools and techniques which we think relevant users would realistically have access to. In practice we perform capabilities elicitation once, testing as wide a range of techniques and tools as possible that we think are likely to improve performance.

Our QA evaluations involve posing open-ended questions and assessing the responses based on detailed grading guidance. Although this approach is more difficult than grading multiple-choice answers, as detailed by Biderman et al. , we rely on open-ended questions more than multiple-choice questions because they:

Responses to our QA evaluations typically range from 100 to 400 words. Grading these responses can be largely automated, enabling QA evaluations to serve as the broadest and most rapid tool in our evaluation suite. For a full assessment of AI system capabilities, QA evaluations would always be combined with other methods such as longer horizon tasks and/or randomised controlled trials.

In this post, we'll outline the five steps AISI follows to develop robust QA evaluations, sharing key questions to consider and lessons we've learned from developing hundreds of QA pairs. Our QA evaluations are run via Inspect , our opensource framework for frontier model evaluations.