Multi-agent systems
Learn how to structure multi-agent architectures with orchestrators and specialists, when parallel subagents save time, and how to handle failures when individual agents go wrong.
TL;DR
- Multi-agent systems pair a coordinating orchestrator with specialized worker agents. The orchestrator decomposes and routes; workers execute and return typed results.
- Parallel execution means latency = max(T_i), not sum(T_i). Ten 30-second tasks finish in 30 seconds, not 300 seconds, a 10x wall-clock improvement for fully independent subtasks.
- Typed agent communication (Pydantic schemas, TypedDict) prevents the most common multi-agent bug: free-text handoffs that propagate malformed data silently into the synthesis step.
- Failure isolation is mandatory. One worker failing must not cancel the entire workflow. Use
return_exceptions=Truein async gather, retry once, then flag partial results. - Cost scales as N_agents × M_tool_calls × token_price. A 5-agent pipeline using GPT-4o at $15/million output tokens can cost $4 to $8 per complex run without per-agent budgets.
- The engineering decision this enables: when a task exceeds one context window or benefits from specialization, replace the overloaded single agent with a typed, budgeted multi-agent pipeline.
The problem it solves
Ask a single agent to "research the top 10 AI startups, analyze each one's competitive strategy, compare their technical approaches, and produce a board-ready report." The agent has two structural problems. First, doing all 10 companies sequentially means total wall-clock time is 10x the per-company time. Second, the same LLM doing raw retrieval, strategic analysis, and formatting is mediocre at all three because they demand different reasoning modes and different tools.
The cost of this in production: a research workflow that should take 3 minutes takes 30. Users abandon it. Engineers add retries that push it to 45 minutes. The underlying cause is not prompt quality; it is architecture.
The math is direct: $T_{sequential} = \sum_{i=1}^{n} T_i$ versus $T_{parallel} = \max(T_i)$. For 10 equally-sized tasks, that is a 10x improvement. The catch is that parallel execution only works when subtasks are independent. Dependency management is the orchestrator's job.
What is it?
A multi-agent system is a collection of individual AI agents coordinated by an orchestrator to complete tasks that benefit from specialization or parallelism. Each agent has its own system prompt, tool set, and context budget. The orchestrator manages the flow of work between them.
Think of it like a project manager with a team of specialists. The project manager does not write code, conduct interviews, or design the database schema. They decompose the project, assign work to the right specialist, monitor progress, and integrate the outputs into a coherent result. Each specialist excels at their narrow domain because they focus on it exclusively.
The fundamental distinction from a single agent in a loop: coordination and execution are handled by different agents. The orchestrator reasons about what to do next; workers do it.
How it works
Orchestrator + worker pattern
The orchestrator receives a top-level task, breaks it into subtasks, and routes each to the appropriate specialist worker using a meta-tool: "dispatch task to agent X with this typed input." Workers are purpose-built agents with system prompts scoped to their specialty, tools that match their domain, and context budgets limited to their subtask.
Continue Reading with Premium
Unlock this article and every other in-depth system design guide on the platform with NotesFromSDE Premium.
Related Articles
Learn how the ReAct loop works, what tool use looks like under the hood, and why compound failure math is the central challenge every production agent team faces.
Learn how LangGraph models agent state as a typed graph, how conditional edges enable complex branching workflows, and how persistent checkpointing lets agents survive crashes and support human approval gates.
Learn why production agents fail when demos succeed, how to reduce blast radius through sandboxing and cost limits, and what reliability patterns make AI agents safe to deploy.
Learn when AI agents need human approval gates, how to implement pause-and-resume in LangGraph, and how to calibrate the approval threshold to balance safety with autonomy.