How do frontier AI agents perform in multi-step cyber-attack scenarios?
We tested seven large language models (LLMs) on two custom-built cyber ranges, measuring their ability to execute extended attack sequences in complex environments. If AI agents can reliably execute multi-step attack chains with minimal human oversight, it could lower the skill barrier for unsophisticated threat actors, increase the sophistication of attack achievable by experience ones, or enable entirely novel offensive operations. To address this gap, we have begun evaluating models on cyber ranges : simulated network environments comprising multiple hosts, services, and vulnerabilities arranged into sequential attack chains; built by cybersecurity experts.
The Core Development
Can AI agents conduct cyber-attacks autonomously? If AI agents can reliably execute multi-step attack chains with minimal human oversight, it could lower the skill barrier for unsophisticated threat actors, increase the sophistication of attack achievable by experience ones, or enable entirely novel offensive operations.
As cyber capabilities improve, increasingly sophisticated testing is needed to accurately measure them. Existing cyber evaluations rely on isolated capture-the-flag (CTF) challenges or question-answer sets. While valuable for measuring specific skills, these approaches don't capture whether AI systems have the autonomous, long-horizon capabilities required for executing extended attack sequences in complex environments.
To address this gap, we have begun evaluating models on cyber ranges : simulated network environments comprising multiple hosts, services, and vulnerabilities arranged into sequential attack chains; built by cybersecurity experts.
Real-World Use Cases & Applications
Overall, our results show that cyber capabilities are improving rapidly, making continuous, rigorous testing essential. The sophistication of evaluation environments needed to accurately measure these capabilities is rising in tandem, meaning cyber ranges will form a crucial part of our approach going forward. Future work could focus on building out a broader suite of ranges to test AI capabilities in different environments or even validating performance in real-world settings featuring active defenders.
Deployment & Availability
This matters because scaling inference-time compute requires no specific technical sophistication from the operator. Unlike custom scaffolding, expert prompting, or tailored tooling, anyone can simply give a model more tokens to work with. At current pricing, a 100M-token attempt with Opus 4.6 costs approximately $80 USD. Prior work on isolated cyber tasks has shown that increased evaluation budgets reveal higher success rates; our results extend this finding to multi-step attack chains.
Strategic Perspective & Ecosystem Impact
By comparing seven models released over an eighteen-month period (August 2024 to February 2026) at varying inference-time compute budgets, we observe two capability trends.
First, each successive model generation outperforms its predecessor at fixed token budgets: on our corporate network range, average steps completed at 10M tokens rose from just 1.7 (GPT-4o, August 2024) to 9.8 (Opus 4.6, February 2026). The best single run completed 22 of 32 steps, corresponding to roughly 6 of the estimated 14 hours a human expert would need.
Second, scaling inference-time compute improves performance even further. Increasing from 10M to 100M tokens yields gains of up to 59%, echoing previous findings from AISI on the relationship between cyber capabilities and inference scaling.
Why it matters
We tested seven large language models (LLMs) on two custom-built cyber ranges, measuring their ability to execute extended attack sequences in complex environments Reported by AISI Blog on March 16, 2026.