OpenAI’s article outlines how ChatGPT defends against prompt injection, describing the shift from simple string overrides to sophisticated social engineering tactics. The company notes that early attacks involved editing external content, such as Wikipedia, to embed direct instructions for agents. Modern attacks, reported in 2025, combine social engineering with attempts to extract and transmit private data, achieving success rates of about 50% in tests. To counter these threats, OpenAI employs a combination of source‑sink analysis and a new Safe Url system. Safe Url monitors conversations for content that could be sent to a third party and either prompts the user for confirmation or blocks the transmission. Similar safeguards are applied to navigations in Atlas, Deep Research, ChatGPT Canvas, and ChatGPT Apps, where sandboxed functional applications detect unexpected communications and request user consent. The article emphasizes that designing agents with constrained capabilities—mirroring human customer‑service protocols—helps limit the impact of manipulation. OpenAI continues to refine its security architecture and training to address evolving social engineering risks in AI systems.
Designing AI agents to resist prompt injection
OpenAI explains how ChatGPT defends against prompt injection by treating it as a form of social engineering. The company combines traditional security methods with a new Safe Url system that detects and blocks or confirms potentially dangerous third‑party transmissions. The approach limits the impact of manipulation even when some attacks succeed.
Why it matters
As AI agents gain more autonomy, protecting them from manipulation is critical for user trust and regulatory compliance. Robust defenses reduce the risk of data leaks and malicious behavior in commercial deployments.
Topic dossiers OpenAI