How our Control Red Team is stress-testing frontier monitors
AISI’s new Control Red Team applies adversarial testing to the internal monitors used by frontier AI companies such as OpenAI and Anthropic. The team evaluates whether these monitors can detect autonomous agents that deviate from user requests and take harmful actions. Early experiments using evolutionary search algorithms have uncovered gaps in monitor prompts, LLM weaknesses, and hard‑coded routing rules that allow attacks to slip past detection. The work highlights open research questions about automated attack discovery and the robustness of control safeguards.
The Core Development
As LLM agents become more autonomous, they have more opportunities to cause harm. To mitigate risk, frontier AI developers, such as OpenAI and Anthropic , are already deploying agents under the watch of a monitor: a separate LLM that reviews the agent’s actions and flags them if they are dangerous. The growing field of AI control research aims to design these kinds of safety measures, and evaluate whether they would be robust even if agents were trying to evade detection.
For over two years, AISI’s Red Team has been evaluating misuse safeguards : protections against humans deliberately eliciting harmful actions from a model. We break these safeguards so that developers can find and patch their flaws. We've now spun up a new subteam – the Control Red Team – to bring the same adversarial approach to control safeguards, which address harmful actions that agents take of their own accord. Since these safety measures are often the last line of defence against harmful actions from autonomous agents, stress-testing them is especially important: we strengthen them by identifying vulnerabilities before misaligned agents do.
This blog briefly describes our monitor red teaming exercises, our attempts to automate the search for attacks, and the research questions we need to answer to strengthen our evaluations.
Strategic Perspective & Ecosystem Impact
Typically, frontier AI developers use safety measures that centre on a monitor. Safety depends on when the monitor flags an action and what happens to flagged actions – but an action that is not caught by the monitor cannot be blocked.
Red teaming a control monitor means asking ‘if an AI attacker completed a harmful objective instead of a user request, would the monitor flag the attack?’ For example, if a user asked an agent to fix a bug in their codebase, but the agent instead downloaded and executed some malware, would the monitor flag the attack?
We have already tested, and are helping to improve, the internal control monitors of two frontier AI developers:
Why it matters
As LLM agents gain autonomy, robust control monitors become critical for preventing misaligned behavior. Stress‑testing these safeguards helps developers patch vulnerabilities before real-world deployment.