An evaluation framework for AI misuse in fraud and cybercrime
We developed a scalable approach to measuring how text-based AI models can assist in three complex fraud and cybercrime scenarios. In total, we ran more than 20,000 evaluations across 14 LLMs, with varying capabilities and safety guardrails, to identify which factors most influence fraud misuse assistance. The text-based AI models we tested currently provide limited practical assistance for fraud. Open-weight models that were fine-tuned to remove safety guardrails (i.e., “uncensored” models, widely available on open‑weight platforms) provided substantially more harmful assistance than closed‑weight, safety aligned models, which consistently refused harmful requests.
The Core Development
Incidents of AI-enabled fraud and cybercrime are on the rise. Criminals have reportedly used large language models (LLMs) to secure remote jobs under false identities, profile victims, and craft sophisticated phishing campaigns.
Despite these reports, it’s difficult to systematically measure how useful LLMs are for the planning and execution of these crimes. Fraud is often deliberately concealed, and reports do not always specify which models were used or how well existing defences performed.
We developed a set of long‑form tasks (LFTs) that mirror how fraud and cybercrime unfold in the real world. Unlike most existing evaluations which focus on single-turn prompts, LFTs simulate the multi‑step interactions that real malicious actors use, providing a more realistic picture of misuse risk. In this blog post, we outline our evaluation design and results.
Real-World Use Cases & Applications
We selected three misuse scenarios based on their real-world prevalence and potential impact: romances scams , CEO impersonation , and identity theft . Each scenario included three adversary profiles: non‑technical individuals, technically skilled individuals, and organised groups, reflecting the range of actors who might attempt to misuse AI. Across these tasks, we assessed model responses using rubrics developed with operational law‑enforcement experts, focusing on two dimensions:
Strategic Perspective & Ecosystem Impact
In total, we ran more than 20,000 evaluations across 14 LLMs, with varying capabilities and safety guardrails, to identify which factors most influence fraud misuse assistance. Our study focused exclusively on text-based models – it should be noted that AI-generated images, audio, and video may provide different levels of uplift that were not measured here.
The text-based AI models we tested currently provide limited practical assistance for fraud. Across the full set of evaluations, 88.5% of responses scored low on actionability, and 67.5% scored low on information access. In practice, this means most outputs either declined to answer or offered only high‑level information, creating the same level of risk as if the actor had used a basic internet search.
Only a very small proportion (5 to 7 percent) produced material that looked ready to use or showed signs of specialised insight. These cases were almost entirely concentrated in open‑weight models with weakened or removed safety guardrails.
Why it matters
We developed a scalable approach to measuring how text-based AI models can assist in three complex fraud and cybercrime scenarios Reported by AISI Blog on February 26, 2026.