AI Agent Architecture: The Production Map

Start your AI agent architecture with a single agent that can investigate a case, propose an action and stop for approval. Buy the model and standard infrastructure; build the business rules, permission boundaries and evidence that the job was completed correctly.

A support agent becomes useful when it can connect a customer's message to the right account, policy and action. It becomes dependable when the surrounding application can explain what it accessed, prevent an unauthorized change and recover when a tool fails.

Think of the system as a transit network. The model chooses a route, tools reach destinations, memory preserves the journey, and orchestration manages movement. Guardrails and approval gates control access; evaluation and logs tell you whether passengers reached the right place.

The component descriptions below were verified against Anthropic, OpenAI, Google Cloud and the Model Context Protocol specification on 25 October 2026. The support workflow is an illustrative design, not a claim about a deployed customer system.

Agent Architecture in AI: What the System Owns

The architecture decides what the model may choose and what the application must enforce. That boundary matters more than the framework name.

Anthropic's augmented LLM, a language model connected to retrieval, tools and memory, is a useful starting component. Its guidance distinguishes a workflow whose route is prescribed in code from an agent that chooses its next actions dynamically; it recommends adding complexity only when the task needs it. Anthropic's agent guidance

For support triage, the model might decide whether to search a troubleshooting guide or inspect an order. Application code should still determine which customer's records it can read, whether a proposed refund is allowed, and what happens if the run stops halfway through.

Keep predictable work predictable. Authenticating an account, checking an entitlement and recording a completed action do not need an agent to improvise. Let the model handle the uncertain part: understanding the message, selecting relevant evidence and proposing a useful next step.

AI Agent Architecture Diagram: The Layers and Connections

An agent needs a controlled loop, with persistent state and checks around it. The layers below are responsibilities; they do not each require a separate service or subscription.

Read the connections as a loop: the application assembles context, the model requests an action, the runtime checks and executes it, and the result becomes context for the next decision. The agent loop guide covers that cycle and its stop conditions in detail.

Model: Interpret the Case and Propose the Next Action

OpenAI Agents SDK, its library for building agent workflows, packages a model, instructions and tools in an Agent; the model interprets the ticket and proposes what to do next. In this design, it receives an account-access problem and requests the relevant entitlement record before drafting a reply. Choose the model by evaluating representative support cases, because eloquent wording does not prove correct tool selection or authority to act. OpenAI agent definitions

Tools and MCP: Reach Business Systems

Model Context Protocol (MCP), a standard interface between AI applications and external capabilities, defines named tools with an inputSchema, discovery through tools/list, and invocation through tools/call. Your account lookup can be exposed through MCP or an ordinary function calling a business API; either way, the backing service must validate arguments and enforce access to the correct account. MCP standardizes the connection, while the MCP explainer covers the protocol and its trust boundaries. MCP tools specification

Memory: Preserve the Right Context

Google's Agent Development Kit (ADK), its framework for building agents, uses Session and state for information within an interaction and supports cross-session memory through components such as Memory Bank. For support, keep the current case and pending approval in application storage, retrieve current account facts from their source, and retain customer preferences only when appropriate. Treat these as distinct jobs; the agent memory guide explains context, session storage and longer-lived memory. Google Cloud architecture guidance

Orchestration: Own Progress, Pauses and Recovery

OpenAI's building-agents guide describes the Agents SDK as handling the tool loop, handoffs and guardrails, which gives your application a runner for model-driven work. Around that runner, persist whether the ticket is investigating, awaiting review, executing or finished, so a restarted process can recover the case. The orchestration guide explains how agent runs fit inside a longer business process. OpenAI's agents guide

Guardrails and Human Approval: Control Consequences

OpenAI Agents SDK separates automatic guardrails from approval interruptions that pause a tool call and return resumable state. For the support agent, validate account access and proposed arguments automatically, then require an authorized person to approve a sensitive change before execution. Put the enforcement beside the action-producing tool; the guardrails guide covers the checks in depth. OpenAI guardrails and human review

Evaluation and Observability: Establish What Happened and Whether It Was Good

OpenAI Agents SDK tracing records model calls, tool calls, handoffs and guardrail activity, while its trace-grading guidance explains how to score workflow behavior against explicit criteria. Observability shows how a support ticket unfolded; evaluation checks whether the agent selected the correct account, followed policy and produced a supported answer. Choose collection and retention deliberately before comparing LLM observability tools. SDK tracing, agent evaluation

Google Cloud's reference architecture assembles ADK, Cloud Run, model access, MCP connections, Model Armor and observability into a concrete deployment. Its documented example explicitly omits memory and session storage from the diagram, so persistent case state remains a design decision rather than something to assume is included. Reference architecture

From Inbound Ticket to Resolved Reply

Give the agent a bounded case with observable completion. In this example, a customer says they paid for access but still cannot use the service; the agent can investigate and prepare a correction, while a support lead approves account changes and the outgoing reply during the initial rollout.

The tool names below describe application functions you would implement, not a vendor's ready-made support integration.

1. 1

   Accept and identify the case

   The ticket webhook, an event sent by the support system, creates or resumes a case using the ticket's external identifier. Authenticate the customer through the existing account flow; do not let an email address typed into the message establish access to an account. Save the incoming message as untrusted customer content.

2. 2

   Retrieve evidence through narrow tools

   Give the agent capabilities such as get_account_entitlement and search_support_policy. Bind account access to the authenticated identity in application code, and return the relevant record and policy reference. The model can choose which evidence to request without receiving unrestricted database access.

3. 3

   Prepare a supported proposal

   The model compares the customer's report with the account record and applicable policy. If the evidence supports an entitlement correction, it drafts the proposed change and reply; if evidence is missing or contradictory, it asks for clarification or escalates. Store the evidence references with the proposal so the reviewer can inspect its basis.

4. 4

   Pause before the business action

   Validate the proposed operation against policy, then save the pending action and approval state. Show the support lead the affected account, exact proposed change, supporting evidence and customer-facing reply. A rejection keeps the write blocked and returns the case for revision or manual handling.

5. 5

   Execute and verify the result

   After approval, recheck that the account and proposal still match what was reviewed, then call the correction tool. Use an idempotency key, an identifier that prevents the same operation being applied again on retry, where the backing service supports it. Confirm the resulting entitlement before sending a reply that says access was restored; record the send result before marking the case complete.

6. 6

   Turn the outcome into evaluation evidence

   Link the execution result, approved reply and final case status to the trace. Route uncertain outcomes to a person rather than interpreting a timeout as either success or failure. When review finds a mistake, preserve a suitably redacted example in the evaluation set.

The crucial failure is an ambiguous write: the account service may have applied the change even though your application never received its response. Reconcile against the source system before retrying. A transcript saying “I will restore access” is evidence of intent, not evidence that access changed.

AI Agent Architecture Patterns: When Another Agent Earns Its Place

Keep a single agent while the task shares a coherent policy, tool set and owner. Add a specialist when it solves a demonstrable separation problem, then test whether the improvement survives the added coordination.

OpenAI distinguishes handoffs, where a specialist takes over the conversation, from agents as tools, where the main agent calls a specialist but retains responsibility for the reply. Its guidance recommends splitting when instructions, capabilities or policy materially differ. Orchestration and handoffs

For the support case, searching a help article and drafting a reply do not automatically deserve separate agents. A billing dispute may: it can require a different policy, restricted financial tools and a different reviewer. You can call that specialist for a bounded assessment while the support agent remains the case owner, or hand over ownership explicitly when the billing workflow should take charge.

Multi Agent Architecture Diagram: Split at a Responsibility Boundary

Different policy is a useful trigger, not a universal requirement for splitting. Separate evidence-gathering tasks can also justify parallel workers, but only if the final owner can reconcile conflicting results and the observed benefit warrants the extra model work.

Before adding a specialist, write down the failure it should fix: wrong-tool selection, conflicting instructions, excessive context or missing expertise. Compare the proposed design on the same cases, including failure cases. Keep it only if the result improves enough to justify added latency, usage, debugging and review work; this is a decision rule, not a promised performance gain.

Naming agents “researcher,” “critic” and “manager” does not establish independent judgment. If they share the same mistaken evidence, their agreement may only repeat the error. For the broader taxonomy, use the types of AI agents map.

AI Agent System Architecture: What to Buy and What to Own

Buy the interchangeable machinery and own the business contract. For a first production agent, the strongest reason to build a component yourself is that your workflow requires behavior the available component cannot express or expose.

For founders: buy a managed agent product if it supports your actual workflow, approval boundary and evidence export. Build around standard components when account-specific policy or integration behavior is your differentiator. A convincing demonstration is insufficient if you cannot recover a paused case or inspect a failed action.

For CTOs and builders: keep business tools narrow and testable independently of the model. A managed runtime can host the application; it cannot infer the difference between a customer asking for a refund and an employee authorized to issue one. Define that distinction in the service that performs the operation.

For operations leads: own the exception queue and resolution definition. Specify who takes over an unresolved case, what evidence they receive and when a ticket may be closed. An unattended approval inbox is an unfinished workflow.

Code execution is optional for this support agent. If a later task must run generated code, treat that as a separate isolation decision and use the agent code sandbox guide; do not introduce a general shell just to look up an account.

Where a Person Approves, and What to Log From the Start

Place approval immediately before the action that needs delegated authority. Reviewing the agent's prose after an account change has already happened cannot prevent that change.

OpenAI's approval mechanism returns a paused run and resumable state; your application resolves the pending decision and resumes it. The same documentation warns that agent-level input and output checks do not surround every intermediate tool call, which is why the action boundary needs its own validation. Approval lifecycle and workflow boundaries

For the illustrative rollout, the reviewer sees the exact account change and reply together. If the action, recipient or evidence changes after review, require a fresh decision. Store who approved what, which policy applied and whether execution matched the approval.

Use the following as a starting log contract. A trace ties the events of a case together; a span records an individual operation within it.

• Case identity: case identifier, trace identifier, originating event and authenticated account reference. Use references rather than duplicating customer details everywhere.
• Configuration: model identifier returned by the provider, instruction version, tool schema version and policy version. These explain which behavior was in force.
• Evidence: retrieved document identifiers, relevant revisions and access scope. Record what supported the decision without indiscriminately copying private records.
• Actions: tool name, redacted arguments, timestamps, duration, result status and retry or idempotency identifier. Preserve enough information to distinguish a proposed action from a completed action.
• Decisions: validation outcomes, escalation reason, approval status, reviewer reference and the proposal approved.
• Outcome and usage: final ticket status, confirmed side effects, delivery result, provider-reported usage and evaluation labels. Track customer corrections or reopened cases when the support system exposes them.

This is a recommended application schema, not a promise that an SDK collects every field automatically. Restrict access, set retention and redact secrets before export. Do not make hidden model reasoning a logging requirement; observable inputs, outputs, actions and evidence must carry the audit trail.

The Limits That Matter, and What's Overhyped

The most important limit is the authority you delegate. Set explicit ceilings for run duration, tool activity and resource use, plus a clear escalation path; choose the values from your workload and service requirements rather than borrowing an arbitrary limit from a demo.

Memory is not a source of truth. If a saved conversation says the customer has access and the account system disagrees, investigate the discrepancy against current authoritative records. More remembered text does not repair stale evidence.

MCP is not a permission grant. Its tools specification still requires access controls and input validation. A connector that exposes a broad operation needs the same business authorization review as a custom API. MCP security requirements

A completed run is not a successful case. Evaluate whether the correct account was changed, the policy was followed and the reply matches the result. OpenAI's evaluation guidance moves from inspecting traces to repeatable datasets so changes can be judged against consistent criteria. Agent workflow evaluation

Several agents do not eliminate ownership. Someone must own the final answer, shared evidence and unresolved exceptions. A specialist is useful because its responsibility is different, not because the architecture diagram looks more sophisticated.

Questions About Agent Architecture

What is AI agent architecture?

AI agent architecture is the arrangement of model decisions, tools, memory, orchestration, controls and evaluation that lets an application pursue a task. For a business agent, the architecture also determines who may authorize actions and how completion is verified.

What is the concept of AI agent?

An AI agent uses a model to select actions toward a goal, observes the results and decides how to continue. Its useful autonomy is bounded by the tools, permissions and stopping rules supplied by the application.

What are the different types of agent architectures?

A practical distinction is a predefined workflow, a single agent choosing among tools, and collaborating specialists. The right choice depends on how predictable the task is and whether different responsibilities need separate tools, policies or ownership; these are implementation patterns, not a universal taxonomy.

Your Monday Move: Draw the Case Before Choosing the Stack

Start now if you have a repeatable support queue, accessible source records and a person who owns exceptions. Choose a representative case and mark the incoming event, permitted reads, proposed writes, approval point and evidence required for closure. Run the initial workflow in draft-and-review mode, then use observed failures to decide which permissions can responsibly expand.

Wait on autonomous execution if account identity, policy ownership or completion evidence is unresolved. You can still improve retrieval and draft quality while fixing those dependencies. If the workflow already follows reliable deterministic rules and does not need model judgment, keep that automation.

The first deliverable should be a case you can explain from intake to outcome, including what happens when approval is denied or a tool's result is unknown.

For practical guides to building and operating AI systems, join the newsletter.