5 production challenges in multi-agent systems and how Deep Insight solved them

Building a multi-agent system is now pretty straightforward. Taking it to production is where the real challenges begin. Thanks to open source frameworks and cloud services, you can set up an agent in a matter of days. A few tool calls, a few lines of prompt, and you have something that looks impressive. But the moment teams try to move beyond a pilot and into real business applications, they hit a familiar wall. If you’ve ever tried to put an agent into production, these frustrations will sound familiar:

Why doesn’t the agent follow our business logic, skip validation steps, or catch edge cases?

You tweak the prompt, add more examples, swap out the model. The same problems keep coming back. That’s because agents don’t follow a fixed sequence of instructions. They make their own judgments based on context. Instructions alone aren’t enough. You need to design how the agent thinks, when it verifies, and what it does when things go wrong. This is exactly why it’s become critical to go beyond prompt engineering alone and design the agent’s entire cognitive architecture and execution flow.

To tackle these challenges head-on, the AWS Solutions Architecture team built Deep Insight , a production-grade multi-agent system. In this post series, we share what the team learned firsthand while building Deep Insight across three posts. We will walk through how we applied Context Engineering techniques proposed by Anthropic (multi-agent architecture, structured note-taking, human-in-the-loop, and more) in a real production environment. We also cover the trade-offs we encountered along the way (with code) and the operational insights we gained from deploying and running this agent on AWS.

This is Part 1 of a three-part series. It defines the five core challenges any production multi-agent system must solve and shows how each layer of the stack addresses them, from Strands Agents SDK and Amazon Bedrock to Amazon Bedrock AgentCore , Application Load Balancer , and AWS Fargate . Parts 2 and 3 cover Context Engineering and production deployment.

We will explore the architecture in detail later. First, let’s see it in action. For the demo, we feed advertising data from a fictional food company, Yummy Food ( yummy-food-market.csv ) with a column definitions file ( column_definitions.json ), and provide the prompt: Analyze this from a sales and marketing perspective, generate charts and insights, and produce a .docx report. Deep Insight generates an analysis plan, presents it to the user for approval, and, once approved, executes the analysis.

The third is the Managed AgentCore version , a production deployment option powered by Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale, with any framework or model. This version deploys the agents on AgentCore runtime, a capability of Amazon Bedrock AgentCore. The system runs entirely within a private virtual private cloud (VPC), with a custom Code Interpreter hosted separately on AWS Fargate, and agents run concurrently in isolated per-user environments. It’s the right choice when you need session isolation and private VPC deployment. Part 3 covers its deployment and operations in detail.

The Deep Insight web demo video that follows shows the full workflow, from uploading data and reviewing the analysis plan to real-time streaming and downloading the finished report.

The first is the Self-Hosted version , which runs locally through the CLI. It’s optimized for rapid iteration with full code-level access to every agent, prompt, and workflow. It sets up in about 10 minutes, which makes it well suited to development. (A Self-Hosted demo video is available on YouTube .)

The Supervisor’s system prompt defines strict workflow rules:

The Planner uses a Claude Opus model with Extended Thinking turned on to develop the analysis plan. It organizes the plan into a [ ] checklist format that specifies concrete tasks assigned to each agent, stored in a full_plan variable. It breaks the request into 5–10 clear steps, as shown in the following example:

Deep Insight is a multi-agent system that turns complex data analysis into automated insights. Given a CSV file and a natural-language prompt, a hierarchy of specialized agents (Coordinator, Planner, Supervisor, Coder, Validator, Reporter, and Tracker) collaborates on the analysis. Together they load the data, run analyses, generate charts, validate calculations, and produce a professional report (.docx) with citations.

Deep Insight is open sourced on GitHub with a full setup guide, and offers three deployment options spanning development to production.

The second is the Web version , a browser-based UI for uploading data, reviewing analysis plans (Human-in-the-Loop, or HITL), and downloading reports, all without writing code, making it accessible for non-developers. (The preceding demo video shows the Web version in action.)