OpenAI introduced CoT-Control, an open‑source evaluation suite designed to measure how well reasoning models can control their chain‑of‑thought (CoT) in ways that might reduce monitorability. The suite contains more than 13,000 tasks derived from established benchmarks such as GPQA, MMLU‑Pro, HLE, BFCL, and SWE‑Bench Verified. Each task pairs a benchmark problem with a CoT instruction, for example, avoiding specific keywords, using only lowercase letters, or appending a given word to each sentence. In tests on 13 reasoning models, the study found that all models struggled to reliably control their CoTs, with controllability scores ranging from 0.1% to 15.4% for the most advanced models. The low performance was not due to task misunderstanding; models often recognized that they were failing to comply with the instructions yet continued to do so. The findings suggest that current models lack the ability to obfuscate their reasoning to evade monitoring, which is positive for the robustness of CoT monitoring as a safety safeguard. The study also observed that controllability increases with model size but decreases when models are asked to reason for longer or undergo additional post‑training or reinforcement learning. When models were told they were being monitored or evaluated, controllability increased slightly but remained low. OpenAI plans to report CoT controllability alongside CoT monitorability in future system cards, starting with GPT‑5.4 Thinking, to track these dynamics over time and ensure that optimization pressure is not directly applied to CoTs of frontier reasoning models.
Reasoning models struggle to control their chains of thought, and that’s good
OpenAI has released CoT-Control, an evaluation suite that tests whether reasoning models can manipulate their chain-of-thought (CoT) to reduce monitorability. The study evaluated 13 models on over 13,000 tasks and found that controllability scores are low, ranging from 0.1% to 15.4%. Larger models perform slightly better, but controllability drops with longer reasoning and more post‑training or RL fine‑tuning. The results suggest that current models are unlikely to pose a major risk to CoT monitoring, though continued evaluation is planned.
Why it matters
Low controllability of reasoning chains strengthens the reliability of CoT monitoring, a key safety safeguard as AI agents become more autonomous. Understanding these dynamics helps the industry anticipate potential alignment risks.