Measuring tactical intelligence targeting and conventional weapons capabilities of AI models

Anthropic’s Frontier Red Team has developed new evaluations to measure AI capabilities in tactical intelligence targeting, such as finding individuals based on fragmentary information, and conventional weapons development, such as engineering drones to strike moving targets. According to a new report from Anthropic’s Threat Intelligence Team, threat actors are already perceiving benefits from using AI models in surveillance and conventional weapons development. The newly developed capability evaluations show that models are making consistent progress on simulated intelligence and weapons development tasks. Open-weights models tested by the team generally lag behind frontier models, typically performing between Sonnet and Mythos-class levels, but still exhibit concerning capability levels. The research evaluated models on tasks including account linkage and individual classification using simulated social media content across varying difficulty tiers. Mythos Preview emerged as a top-performing model on these account linkage and classification tasks, while other models like Kimi K3 performed similarly to frontier models on easier samples but lagged on more difficult, noisier tasks. Anthropic notes that the speed of models is a notable factor, with Claude Mythos Preview taking about 11 minutes on average to produce assessments for median-length samples that would take a human analyst hours to read and analyze. Looking ahead, Anthropic suggests that AI capabilities will not plateau and could make substantive contributions to novel and geostrategically consequential breakthroughs in military and intelligence domains, affecting how models are trained, safeguarded, and released.