Brian Jagger
I have been deploying AI agents into customer support organisations for the better part of three years, and the shift I am watching unfold in 2026 is unlike anything I saw with chatbots a decade ago. The previous generation of support automation was a flowchart wearing a costume. It could only handle the questions someone had hard-coded in advance. The current generation of AI agents for customer support reasons about the conversation, consults knowledge bases on the fly, executes refunds or order changes through real APIs, and escalates gracefully when the situation calls for a human. The teams I work with are not asking whether to adopt agentic support any longer. They are asking how to architect it so the agent stays accurate, safe, and aligned with their brand.
AI Agents for Customer Support: Architecture & Examples
In this article I walk through the architecture I recommend, the metrics that matter, the vendors leading the market, and the governance patterns that keep refund agents from going rogue. I have included examples from Intercom Fin, Decagon, Sierra, and Forethought, plus a candid view on when AI should fully replace tier one and when it should only augment. By the end you will have a blueprint for designing a support agent that customers actually trust and that your CFO can defend on the cost line.
Why support is the killer use case for agentic AI
Customer support has three properties that make it the cleanest fit for agentic AI. First, the work is high volume and repetitive, which means even a modest deflection rate translates into meaningful headcount savings. Second, the knowledge base is already written down somewhere, even if it is fragmented across Zendesk macros, Confluence pages, and tribal Slack threads. Third, the success criteria are unusually measurable. You either resolved the ticket or you did not, and the customer either rated the interaction well or they did not.
I tell the operations leaders I advise to look at their ticket distribution before they pick a vendor. In most B2C businesses, the top twenty intents account for roughly eighty percent of contact volume. Order status, returns, password resets, billing disputes, and shipping queries dominate. An agent that handles those twenty intents at ninety percent accuracy will deflect more than half of all inbound contacts on day one.
If your support data shows a long tail of unique issues with no repeat structure, the economics of agentic support change. The agent will still help, but the payback period stretches.
The other reason support is special is that customers have come to expect immediate responses. A well designed AI agent answers in seconds, twenty four hours a day, in the customer’s preferred language. That responsiveness alone moves CSAT before you measure anything else.
The reference architecture for a support agent
Every support agent I have built follows the same five layer architecture. The conversation layer accepts input from chat widgets, email, voice, WhatsApp, or in app messaging. The understanding layer classifies intent, extracts entities like order numbers, and decides which workflow applies. The retrieval layer pulls relevant knowledge from a vector database, a structured FAQ, or a product catalogue. The action layer executes API calls into the order management system, the billing platform, or the CRM. The escalation layer detects when a human should take over and packages the conversation context cleanly.
| Layer | Responsibility | Common tools |
|---|---|---|
| Conversation | Channel ingestion and transcript management | Twilio, Front, Zendesk, Intercom |
| Understanding | Intent, entity, sentiment | LLM with structured output, fine tuned classifiers |
| Retrieval | Knowledge grounding | Pinecone, Weaviate, Elastic, Vertex AI Search |
| Action | API execution and state mutation | Internal APIs, MCP servers, workflow engines |
| Escalation | Handoff and audit | Zendesk, Salesforce Service Cloud, PagerDuty |
The reason I separate these layers is testability. When an agent gives a bad answer you want to know whether the failure was in classification, retrieval, generation, or execution. A clean separation lets you instrument each stage and replay traffic against new prompts without retraining the whole stack.
Intent classification and conversation routing
Intent classification is where many teams over engineer. I used to recommend fine tuned BERT classifiers for every intent in the taxonomy. With modern LLMs you can replace most of that with a structured prompt and a JSON schema, then route based on the predicted intent. The accuracy is comparable, the latency is acceptable, and the developer experience is dramatically better.
I keep the intent taxonomy flat and shallow. Three or four top level intents like order, billing, product, account, and account, then sub intents one level down. Deep hierarchies look elegant in a Miro board and fall apart in production. The agent gets confused, the analytics get noisy, and routing rules become impossible to maintain.
For routing, I use a hybrid approach. Deterministic rules handle the obvious cases. If the customer types a refund request and the order is over thirty days old, route to the policy specific subflow. If the message contains a legal trigger word like lawsuit or attorney, route to the human queue immediately. Everything else falls through to the LLM driven planner that decides which tools to call.
The routing layer is also where I implement the policy guardrails. An agent that can execute refunds should have explicit rules about the maximum value it can authorise without human approval. I have seen agents authorise five figure refunds because nobody set a cap.
Retrieval augmented generation for support knowledge
Retrieval is the single biggest determinant of answer quality. I have rebuilt the retrieval layer three times on most projects because the first version always under performs. The standard mistake is to embed your entire knowledge base as raw documents and hope cosine similarity finds the right chunk. It does not.
The retrieval design I now use looks like this. Each document is chunked semantically by section, not by token count. Each chunk gets enriched metadata, including product, region, language, and an intent label. Retrieval runs as a hybrid search combining BM25 keyword matching with dense vector similarity, then reranks the top fifty results with a cross encoder. The agent receives the top five reranked chunks with their source URLs.
- Chunk by semantic section, not by character count
- Tag every chunk with product, region, and language metadata
- Use hybrid search with reranking, not pure vector similarity
- Always pass source URLs through to the response so customers can verify
- Refresh embeddings whenever the source content changes, not on a monthly cron
I once watched a support agent confidently quote a refund policy from a document that had been superseded eighteen months earlier. The vector store had stale chunks. The customer was furious. The fix was a content freshness pipeline, not a smarter model.
Action tools, APIs, and write operations
This is where agentic support pulls away from the chatbot generation. A real agent does not just answer questions. It looks up your order, checks your subscription status, applies a credit, swaps out a damaged item, and confirms the change. Each of those capabilities is exposed as a tool the agent can call.
I implement tools as small, focused functions with strict input and output schemas. A tool that issues a refund accepts an order ID, an amount, and a reason code. It returns a confirmation ID or a structured error. The tool itself enforces business rules, so the agent cannot bypass them by being clever about prompting.
Write operations need extra care. I categorise tools as read, soft write, and hard write. Read tools are safe to call freely. Soft writes can be reversed within a short window. Hard writes are irreversible, like issuing a refund or cancelling a subscription. Hard writes always require a confirmation step, either from the customer or from a human supervisor.
The Model Context Protocol has made tool integration significantly cleaner in 2026. I now expose backend systems through MCP servers, which means the same tool definitions work across multiple agents and frameworks. That saves a remarkable amount of duplicated glue code.
Escalation, handoff, and the human in the loop
Escalation design separates the products that customers love from the ones they tolerate. A bad escalation feels like restarting the conversation with a stranger. A good escalation feels like the agent introduced you to a colleague who was already briefed.
The escalation triggers I always implement are sentiment based, complexity based, and explicit. Sentiment based escalation fires when the customer expresses frustration, repeats the same question, or uses profanity. Complexity based escalation fires when the agent’s confidence drops below a threshold for two consecutive turns, or when the issue requires authority the agent does not have. Explicit escalation fires whenever the customer asks for a human, full stop.
When escalation fires, I package the conversation as a structured summary for the human agent. That summary includes the customer’s stated problem, the actions the agent already took, the relevant knowledge base articles, and the agent’s best guess at next steps. The human agent should be productive within fifteen seconds of picking up the case.
| Escalation trigger | Latency target | Handoff payload |
|---|---|---|
| Sentiment red flag | Under 5 seconds | Transcript plus sentiment scores |
| Explicit human request | Immediate | Transcript plus customer profile |
| Confidence collapse | Under 10 seconds | Transcript plus failed tool calls |
| Policy boundary hit | Immediate | Transcript plus policy reference |
CSAT, deflection, and the metrics that actually matter
I have watched executives obsess over deflection rate to the exclusion of everything else. Deflection is important. It is not the whole story. The metrics I run on every support agent dashboard cover quality, cost, and customer experience.
Quality metrics include resolution rate, accuracy on a holdout evaluation set, and the rate at which the agent invents facts. Cost metrics include cost per resolved contact, average tokens per conversation, and tool call counts. Customer experience metrics include CSAT, time to resolution, and the rate at which customers come back about the same issue within seven days.
The metric I watch most carefully is repeat contact rate. A deflection that produces a satisfied customer in one turn is genuine value. A deflection that produces a frustrated customer who calls back three times is a hidden cost. I have seen reported deflection of seventy percent collapse to forty percent net deflection once you correct for repeat contacts.
- Resolution rate by intent, not in aggregate
- Cost per resolved contact, including LLM and infrastructure
- CSAT split between agent handled and human handled
- Repeat contact rate within seven days
- First response time and time to resolution
Intercom Fin, Decagon, Sierra, and Forethought compared
The vendor landscape in 2026 has consolidated around a few capable platforms. I have shipped production deployments on all four and they each have a personality.
Intercom Fin is the most polished out of the box. It plugs into Intercom’s existing data model, learns from your past resolutions, and gets to acceptable accuracy quickly. It is the right choice for teams already on Intercom that want to ship in weeks rather than months. The trade off is limited flexibility on custom workflows.
Decagon takes a more enterprise oriented approach. The platform is built around the idea of agent operating procedures that mirror how a human team would handle each case. It works well for organisations with complex policies, like regulated industries.
Sierra, founded by Bret Taylor, has invested heavily in voice and brand fidelity. The agents feel notably more human than the competition, which matters in premium consumer brands. The pricing reflects that, and the integration work is non trivial.
Forethought has been in the market the longest and shows it. The product is feature complete, the analytics are excellent, and the deployment process is mature. Newer entrants outpace it on raw LLM capability.
My usual recommendation is to run a structured bake off with at least two vendors over two months, using real ticket traffic and a shared evaluation set.
Evaluating a support agent before you ship it
I refuse to ship a support agent without a proper evaluation harness. The harness has three components. A curated golden set of one hundred to three hundred representative conversations with ideal responses. An automated grader that scores accuracy, tone, and policy compliance. A human review queue that samples production traffic weekly.
The golden set takes effort to build. I usually spend two weeks sitting with support leads, mining historical tickets, and labelling them with categories like factual accuracy, escalation appropriateness, and tool use correctness. Once built, the golden set runs on every prompt change and every model upgrade. If a change drops accuracy by more than two percent on any category, it does not ship.
LLM as judge has matured enough in 2026 that I trust it for automated scoring on most criteria. I still pair it with human review for edge cases, particularly anything touching legal, financial, or safety claims. The cost of an automated grader running thousands of evaluations is now trivial compared to the cost of a single regulatory incident.
When AI agents replace tier one versus augment agents
This is the question every operations leader asks, and the honest answer is that it depends on the risk profile of the work. Pure replacement makes sense when the intents are well structured, the actions are reversible, and the cost of an occasional error is bounded. Order status, password resets, and information requests fit cleanly.
Augmentation makes sense when the work involves judgement, regulatory risk, or significant financial impact. Insurance claims, account closures, and dispute resolution should usually run with the agent drafting the response and a human reviewing before it goes out.
I have started to think of it as a spectrum rather than a binary. On one end the agent handles everything autonomously. On the other end the agent is a copilot for the human. In the middle the agent handles routine cases and escalates the rest. The right point on the spectrum is set by your tolerance for error and the leverage you need from the team.
Governance for refunds, credits, and irreversible actions
When an agent can move money, governance stops being optional. The governance framework I implement covers authorisation limits, audit logging, anomaly detection, and human override.
Authorisation limits set the maximum value the agent can refund, credit, or write off without escalation. I start conservatively, often at twenty five dollars per transaction, and raise the limit as I gain confidence in the agent’s judgement. Audit logging captures every tool call with the prompt, the response, the customer ID, and the outcome. Anomaly detection flags unusual patterns like a sudden spike in refund volume or the same customer receiving multiple credits in a short window.
Human override is the safety valve. Every action the agent takes should be reversible by a human within a reasonable window, and every customer should have a clear path to escalate a decision they disagree with. The agent is not the final authority, even when it acts autonomously.
I have seen organisations skip governance because the agent was performing well in testing. The first time a prompt injection attack tricks the agent into authorising a fraudulent refund, those organisations regret the omission.
Voice agents and the telephony stack
Voice support has finally caught up to chat in 2026. The combination of low latency speech to text, capable LLMs, and natural sounding voice synthesis means a phone call with an AI agent is no longer obviously artificial. The architecture mirrors the chat stack with extra layers for audio processing and barge in handling.
The vendors I see deployed are Sierra, Vapi, Retell, and Bland. Each handles the orchestration of speech to text, LLM reasoning, and text to speech. The integration challenges sit in the telephony layer. Routing inbound calls, handling DTMF, and integrating with existing IVR systems takes more work than the demo videos suggest.
Latency budgets are tighter in voice than chat. I target under eight hundred milliseconds end to end response time, which constrains how many tools you can call before responding. The pattern I use is to acknowledge immediately, defer expensive lookups to a parallel track, and stream the answer as it becomes available.
Multilingual support and tone calibration
Most enterprise support agents need to handle multiple languages. The modern approach is to let the LLM handle translation natively rather than running through a separate translation service. Quality is now strong enough across English, Spanish, French, German, Portuguese, and Japanese that the round trip approach is rarely worth the latency cost.
Tone calibration is harder than translation. A formal Japanese tone is not the same as a casual American tone, and your brand voice probably sits somewhere specific on that spectrum in each market. I write tone guidelines into the system prompt, with examples of acceptable and unacceptable phrasings, and I evaluate tone separately in the golden set.
The failure mode I have seen most often is inconsistent tone across languages. The agent sounds warm in English and formal in German because the LLM defaults differ. The fix is explicit instruction and language specific evaluation samples.
Common failure modes I have seen in production
I keep a running list of failures so I do not repeat them. The top recurring patterns are stale retrieval, missing tools, over confident responses, prompt injection, and unhealthy reliance on the model’s defaults.
Stale retrieval happens when content changes but the embeddings do not refresh. The fix is a content freshness pipeline that listens for source changes and reindexes within minutes. Missing tools happens when the agent attempts to take an action that no tool supports and confabulates a response. The fix is clear capability boundaries in the system prompt plus a fallback that says explicitly when the agent cannot help.
Over confident responses appear when the agent has no information but generates a plausible answer anyway. The fix is calibrated confidence and a system prompt that prefers I do not know over speculation. Prompt injection is the security failure where a customer message manipulates the agent into ignoring its instructions. The fix is input sanitisation, output validation, and treating customer messages as untrusted data throughout the stack.
A 90 day rollout plan for an enterprise team
The rollout plan I run with enterprise clients has three phases. Days one to thirty are foundation. Days thirty one to sixty are pilot. Days sixty one to ninety are scale.
Foundation covers tooling selection, knowledge base preparation, integration scoping, and evaluation set construction. By day thirty you have a working prototype on a single high volume intent, end to end tests passing, and stakeholder alignment on success metrics.
Pilot is when the agent meets real customers. I shadow first, where the agent generates responses but humans send them. Then I run a limited live test, often ten percent of inbound traffic on one channel. By day sixty you have production data on accuracy, CSAT, deflection, and cost, and you have iterated on the failure modes.
Scale is the ramp from ten percent to majority traffic. The work in this phase is operational, not technical. You are training the human team on new workflows, calibrating the escalation thresholds, and instrumenting the dashboards that the operations team will live in.
