LLM judges on trial: A new statistical framework to assess autograders

Evaluating AI models is central to improving their performance, as well as ensuring their safety. Evaluations let us compare performance across various domains (such as mathematics, coding or reasoning tasks) and assess model usefulness for real-world use cases. They are also important for knowing whether models could possess potentially dangerous capabilities, such as the capacity to assist with – or even autonomously complete – sophisticated cyber attacks, or self-replicate, evading human oversight and control.

Model evaluations can be carried out by humans – a model is prompted to complete a task, and a human grades its performance. But this approach is expensive and time-consuming. As AI progress has accelerated and the volume of evaluations has grown, researchers have sought to automate the process by having Large Language Models (LLMs) grade model outputs. These LLM evaluators are known as autograders , or “LLMs-as-judge”.

This raises a crucial question: can we rely on the judgements of autograders? Some studies suggest they exhibit certain systematic biases, which appear distinct from random noise. For example, they tend to grade outputs from models of the same family more highly. They can also have preferences for longer outputs, or ones written in a particular style or containing certain keywords. This means that before we can confidently use an autograder for AI model evaluation, we need to evaluate the reliability of the autograder itself for the specific task at hand.

What if we could do both at the same time? In a new paper , we propose the use of Bayesian generalised linear models (GLMs) to do just this. GLMs allow researchers to use autograders for model evaluations, while simultaneously assessing the quality of those autograders.

This blog explains why Bayesian GLMs are useful in the context of AI model evaluations. All the statistical models discussed in our paper are available in an open-source package on GitHub .

A GLM is a statistical model that lets us predict an outcome based on weighted input variables. It uses a link function to represent this outcome in a scale that is suitable for the properties of our data . Using a Bayesian approach returns a posterior distribution (a range of possible values and their respective probabilities), as opposed to point estimates.