Measuring political bias in Claude

We want Claude to be seen as fair and trustworthy by people across the political spectrum, and to be unbiased and even-handed in its approach to political topics.

In this post, we share how we train and evaluate Claude for political even-handedness. We also report the results of a new, automated, open-source evaluation for political neutrality that we’ve run on Claude and a selection of models from other developers. We’re open-sourcing this methodology because we believe shared standards for measuring political bias will benefit the entire AI industry.

When it comes to politics, people usually want to have honest, productive discussions—whether that’s with other people, or with AI models. They want to feel that their views are respected, and that they aren’t being patronized or pressured to hold a particular opinion.

Another way to engender even-handedness in Claude is through character training, where we use reinforcement learning to reward the model for producing responses that are closer to a set of pre-defined “traits”. Below are some examples of character traits on which we have trained models since early 2024 that relate to political even-handedness:

We tested the models using 1,350 pairs of prompts across 9 task types and 150 topics. We included prompts of the following categories in our evaluation: reasoning (argue that…), formal writing (write a persuasive essay…), narratives (write a story…), analytical question (what research backs up…), analysis (evaluate the evidence for…), opinion (would you support…), and humor (tell me a funny story…). Our evaluation set not only covers arguments for and against political positions but also ways in which users with different political leanings might ask Claude models for help.

If AI models unfairly advantage certain views—perhaps by overtly or subtly arguing more persuasively for one side, or by refusing to engage with some arguments altogether—they fail to respect the user’s independence, and they fail at the task of assisting users to form their own judgments.

On our own platforms, we want Claude to take an even-handed approach when it comes to politics: 1

One concrete way that we try to influence Claude to adhere to these principles is to use our system prompt—the set of overarching instructions that the model sees before the start of any conversation on Claude.ai . We regularly update Claude’s system prompt; the most recent update includes instructions for it to adhere to the behaviors in the list above. This is not a foolproof method: Claude may still produce responses inconsistent with the descriptions in the list above, but we’ve found that the system prompt can make a substantial difference to Claude’s responses. The exact language in the system prompt can be read in full here .