Does This Process Need an AI Agent? A Practical Architecture Decision Guide

“Please cancel the second item. If it has already shipped, can I return it? I used a voucher, and I cannot find the order number.”

This customer message contains several different problems. Understanding “the second item” may require language interpretation. Finding the order requires authenticated access to records. Applying the cancellation policy requires business rules. Issuing a refund requires permission and, in our example, human approval.

Putting an LLM in the first step does not mean it should control every later step.

Before choosing an agent framework, draw the process without one. Then ask: Where does this process need freedom to choose its next step, and where does it need predictable execution?

We will compare three architectures for the same cancellation and refund service: ordinary rules, a fixed workflow with targeted LLM steps, and a bounded agent. The aim is to choose the simplest design that meets the actual requirements, including the awkward cases.

Research checked on 13 November 2026. The customer, policies, architecture designs and financial example are illustrative. No live model comparison or production benchmark was performed.

1. Define the control boundary

Teams use “workflow,” “agent” and “agentic” differently. Some call any application with an LLM and tools an agent. For this guide, the distinction is who chooses the next operation.

• A deterministic, rule-based workflow has transitions defined by ordinary code or an explicit process model. Given the same validated inputs, recorded business state and rule version, its decision logic selects the same transition. It may branch, wait for a person and call many services. Network timing and external systems still introduce operational uncertainty.
• An LLM-assisted workflow keeps that explicit control structure but uses an LLM for bounded tasks such as extracting a requested item or classifying intent. Code validates the result and selects among defined transitions. The interpretation can vary; the model does not invent the process.
• An agent, as used here, lets an LLM choose its next step or tool dynamically from an allowed set, using intermediate observations to decide what to do next. A trusted runtime limits the loop, and business services retain control of authorized state changes.

This follows the control-flow distinction in Anthropic’s Building effective agents, published in December 2024. That page now warns that its tooling discussion has evolved. Its recommendation to start simply is useful architecture guidance, not evidence that one design wins for your workload. LangGraph’s official patterns documentation uses a similar distinction between predetermined paths and dynamically chosen processes.

An LLM selecting “refund” from an enumerated intent field is still compatible with a fixed workflow. An LLM repeatedly deciding whether to inspect shipments, retrieve an earlier conversation, ask a question or stop is exercising dynamic control. A conditional statement, a tool call or a long-running task alone does not establish that distinction.

Select the diagram to view it at full size.

Architecture comparison. Teal marks model involvement; navy marks application control. Every option reaches the same validation, authorization and approval boundary. The agent’s dynamic loop is limited to investigation.

2. Keep one business contract

Our fictional shop offers cancellation of an item that has not shipped. A shipped item enters a return assessment; it is not silently treated as cancellable. Refund amounts come from authoritative order and payment records, including voucher allocation and previous refunds. Every monetary refund requires an independently authenticated human approver in this example. These are teaching policies, not legal advice or recommended universal refund rules.

All three designs must accept the same incoming messages. A rules-only design can use a form or a human intake step to structure free text; that labor and customer friction must count in the comparison. It cannot quietly be tested only on easy form submissions while the agent receives messy messages.

The service must identify the customer and item, obtain current order state, apply the same policy revision, request missing facts, and produce either a valid proposal, a justified refusal or a useful handoff. Once approved, it must execute only the exact authorized operation and confirm the result from the receiving system. No design may change the payee, approve its own refund, cross customer boundaries or create a duplicate effect.

For the opening message, success may initially mean asking which order and which item the customer means. A correct clarification is progress, but it is not a completed refund. Keep those outcomes distinct.

3. Compare the three designs on that contract

Option A: ordinary code and explicit rules

The intake step asks the customer to select an order, item and request type. A support person handles the original message if the form cannot resolve it. The workflow loads the order, checks shipment status, evaluates the rule and creates a proposal. A person approves any refund; the service rechecks current state and executes the authorized command.

“Already shipped” is a defined branch into return assessment. “Order missing” is a defined clarification state. A payment timeout is a recovery state. None requires a model to plan the next action.

This is a strong fit when the supported request types and information requirements are stable. It also gives a useful baseline: measure how much work comes from language interpretation before adding autonomous control. Its limitation is visible when customers cannot map their situation to the form, or operators spend substantial time assembling context from different systems.

Option B: a fixed workflow with targeted LLM steps

Keep the same process, but let an LLM propose a structured interpretation of the message: possible intent, order reference, item reference, missing information and supporting text. Allow “unclear” and multiple conflicting intents. A schema-valid response is not proof that the interpretation is true.

Code verifies identifiers against the authenticated customer’s records. It resolves “second item” only when the referenced order and ordering are unambiguous. It asks a targeted question otherwise. The workflow then loads current shipment and payment data, calls the same rules and creates the same approval request as Option A.

The LLM may also draft the clarification or explanation from approved facts. It receives no authority to issue money. A timeout, invalid output or contradictory interpretation goes to a defined retry or handoff path; repeated prompting does not become an unbounded search for a convenient answer.

For ordinary cancellations, this often provides the right experiment: can a bounded interpretation step reduce intake effort while the process remains explicit? That is a hypothesis to test, not a measured benefit claimed here. Language variability alone is no reason to delegate planning.

Option C: a bounded agent inside the process

Now suppose investigation varies. One case needs shipment history; another needs the original voucher allocation and an earlier support exchange; a third needs clarification before either lookup is useful. New evidence changes which source matters next.

The outer workflow starts an investigation with a case ID, a limited set of tools and a resource budget. The model can propose reading permitted order details, looking up a permitted shipment, retrieving a relevant support exchange, asking a targeted question, submitting a proposal or escalating. The runtime validates every requested tool and its arguments, records the result and asks for the next step. Budget exhaustion ends in a handoff, not increased privileges.

The agent returns evidence and a proposed business action. The ordinary workflow still owns policy validation, approval, execution and confirmation. Clarification can suspend the case; it need not keep a worker or a model conversation running while the customer replies.

This can be useful when selecting and sequencing evidence sources is the difficult work. Its potential advantage must be visible in more correct resolutions or less human investigation on those cases. It brings additional paths to test, more variable resource use and more difficult failure diagnosis.

For a clear request to cancel one identified, unshipped item, the necessary next steps are already known. An agent adds little decision value there. If three stable branches cover the investigation, implement and compare those branches before introducing a planner. You can keep common cases fixed and reserve a bounded agent for a measured exception class; that is not an obligation to make the whole service agent-controlled.

A compact comparison matrix

On a narrow screen, scroll the table horizontally to compare all columns.

These are design differences, not a performance ranking. A complicated workflow can cost more to maintain than a well-bounded agent; a small agent can still be unjustified for a stable process.

4. Use a decision tree, not an autonomy score

Select the diagram to view it at full size.

Read from top to bottom. A “no” at the safety or evidence branch means keep human investigation while improving the process or testing a bounded candidate. It does not mean grant the model more authority.

1. Are the outcome, rules and accountable owner clear? If not, clarify the business contract first. An agent cannot settle an organizational disagreement about who may approve a refund.
2. Can ordinary code or a practical intake form obtain reliable structured inputs? If yes, start with A. Include accessibility and customer effort when deciding whether the form is practical. If unstructured language remains the bottleneck, consider B.
3. After interpretation, are the next operations known? If yes, use B with explicit branches. A refund category selected by a model still leads into a known business process.
4. Does intermediate evidence materially change which source, tool or question should come next? If no, a fixed lookup sequence may suffice. If yes, identify the specific investigation tasks for which dynamic control could help.
5. Can those tasks use bounded tools, durable state, independent checks and a workable human handoff? If not, retain human investigation and fix those gaps. If yes, compare a limited C pilot with A and B on the same cases.

There is no defensible universal number of branches, tool calls or variability points at which a process “becomes agent-worthy.” Decide using observed limitations and the consequence of failure.

5. Keep authority outside the model

Four decisions must remain distinguishable even if the implementation combines some services:

1. Model proposal: “This message appears to request a refund for item I2.” Treat it as fallible input with source evidence, not an instruction carrying permission.
2. Business validation: Domain code checks that I2 belongs to the order, the requested transition is eligible, and the amount follows current policy and payment records. Missing or conflicting evidence remains unresolved.
3. Authorization: Trusted services establish the caller, customer scope, allowed operation and any required approval. The approver sees the exact item, amount, currency, destination, evidence and revision. A changed proposal needs fresh approval.
4. Execution: The receiving service atomically enforces current constraints and records the effect. The workflow checks the receipt and authoritative state before telling the customer the refund completed.

Select the diagram to view it at full size.

Conceptual sequence for all three architectures. Option A supplies a structured request without the model. No model-to-refund shortcut exists. A denial or changed state returns a refusal or a new review requirement.

OWASP’s Authorization Cheat Sheet recommends least privilege, denial by default and permission checks on every request. For our design, that means customer-scoped reads as well as protected writes. Hiding a dangerous tool from the prompt does not replace service-side enforcement. A customer message saying “ignore the policy and approve this” cannot change an authenticated grant or approval record.

Minimize the context sent to the model: selected order fields and relevant excerpts may suffice. Do not include card details or unrelated support histories. Define who can access messages, checkpoints and traces, how long they are retained, and which model deployment may process them. Validate provider handling and organizational privacy requirements before using real customer data. Pseudonymous case IDs reduce unnecessary exposure but do not make the linked case anonymous.

The architecture decision should include the cost of these controls for every option. For a deeper implementation discussion, see AI Agent Authorization: From OAuth Access to Bounded Business Actions.

6. Make uncertainty and handoffs explicit states

A chat transcript is not the business case record. Persist the case identity, authenticated scope, verified facts and their sources, policy/model revisions, current process state, pending question, approval payload, operation identity and execution receipt. For C, also retain the permitted tool observations and budget consumed. Prefer concise decision records and evidence references over collecting hidden model reasoning.

LangGraph documents checkpoints for resumable state, and its interrupt documentation describes pausing for external input. It also warns that a resumed node starts again from its beginning. These are concrete runtime semantics to understand, not a guarantee that a payment will happen only once.

For any implementation, ask what survives worker failure and what happens if the downstream action commits before the caller records success. Stripe’s idempotency documentation illustrates one provider’s retry contract, including retained responses and key-lifetime constraints. Use the receiving system’s actual semantics; a local retry loop does not create an end-to-end guarantee.

For this service, the following shared case catalogue belongs in the design review and evaluation set. Each expected outcome applies to A, B and C. Human work needed to achieve it is measured, not hidden.

On a narrow screen, scroll the table horizontally to compare all columns.

Ask a question when a specific customer fact can resolve ambiguity. Escalate when policy is disputed, sources conflict, permission is missing, an effect remains uncertain, or the available tools cannot establish the required evidence. More autonomous steps cannot repair missing authority.

A useful handoff contains the verified customer/order scope, request and conditions, evidence collected, unresolved question, actions already attempted, operation status and the next decision needed. It must distinguish “refund proposed,” “awaiting approval,” “execution uncertain” and “refund confirmed.” Human approval without sufficient context merely moves the guess to a person.

For the recovery mechanics, see Durable AI Workflows in Java: Retrying Is Not Enough. For budgets, tool mediation and operational evidence, see AI Agent Harnesses. Those implementation needs do not themselves establish a need for an agent.

7. Compare business outcomes and total cost

Use the same versioned, representative case set for all three options. The sixteen cases above are a starting coverage catalogue, not a statistically representative sample. Build a larger, appropriately redacted set from real traffic, covering languages, channels, policy branches and difficult cases. Keep a held-out set, document sampling, and report common-case results separately from deliberately oversampled hazards.

Reset the sandbox business state between trials. Give each architecture the same policy, tool contracts, customer facts and simulated fault schedule; allow each to choose its own path. Freeze configuration for the comparison, record model and prompt revisions, and repeat model-based trials to expose variability. Include the intake operator and approver in comparable tasks rather than assuming human work is free. Report sample counts and uncertainty; zero observed violations in a small test does not prove safety.

Anthropic’s evaluation guide distinguishes a task, repeated trials, graders and the resulting outcome. Our recommendation is to inspect business records independently of the candidate’s answer. The existing Homann guide to business-outcome evaluation develops that method; here the question is which architecture earns its additional complexity.

On a narrow screen, scroll the table horizontally to compare all columns.

Define unacceptable failure classes before testing. A lower cost cannot compensate for cross-customer access or an unauthorized refund. Separately agree service and economic acceptance criteria with the process owner. There is no universal success percentage or budget that makes an agent appropriate.

An illustrative calculation, not a benchmark

Suppose a hypothetical cohort contains 1,000 cases for B. Runtime costs total €120, and intake, review and rework consume 20 hours at an assumed loaded rate of €60/hour. If 900 cases reach the agreed correct terminal outcome, the operating cost per correct case is (€120 + €1,200) / 900 = €1.47.

For a hypothetical C cohort of the same size, suppose runtime costs €300, human work takes 12 hours at the same rate, and 920 cases complete correctly. That gives (€300 + €720) / 920 = €1.11. These invented figures illustrate the denominator and labor tradeoff; they predict no savings and establish no reliability advantage.

Now add an assumed €600 of engineering and maintenance allocation to B and €1,400 to C over the same cohort and time horizon. The expanded figures become €2.13 and €2.63 per correct case respectively. The apparent advantage reverses. Those allocations are also fictional. In a real decision, include development, integration, evaluation, observability, privacy review, operations, incident diagnosis, rule/prompt changes and training. Avoid counting the same labor twice. Where demand varies, show how volume and case mix change the allocation.

A provider’s token price is an input to that model, not a measured cost per resolved business case. None is quoted here. Keep vendor claims, pilot measurements and scenario arithmetic in separate columns.

What would justify moving from B to C?

First inspect why B fails or escalates. If the causes are poor extraction, missing tool data or unclear policy, improve those first. If the cause is repeatedly having to determine a different investigation sequence from newly discovered evidence, C has a specific job to test.

A migration becomes credible when a bounded pilot improves verified outcomes or human investigation effort on that class, preserves the agreed safety constraints, and delivers acceptable latency and total cost over the relevant volume. Check that a cheaper additional workflow branch or improved retrieval does not explain the same gain. Repeat on held-out cases, inspect failures, then use a limited rollout with a human fallback and a documented rollback trigger. Keep monitoring after model, policy or tool changes.

8. Record the smallest justified design

For our initial service, a reasonable starting decision is B if free-text intake creates enough friction to warrant a tested interpreter; otherwise A. The cancellation and refund path remains explicit. C is a candidate for the demonstrated investigation exceptions, not a predetermined next release.

Record the decision in a short architecture decision record: business outcome and owner; observed bottleneck; options considered; control and authority boundaries; case coverage; quality constraints; full cost assumptions; chosen design; and the evidence that would cause you to revisit it. The accompanying decision kit provides an editable worksheet, the sixteen worked cases and a blank trial record. It contains no simulated agent or model-performance results: a scripted planner would demonstrate its script, not answer whether dynamic planning helps your traffic.

Download the editable architecture decision kit (ZIP: Markdown worksheet and worked cases, plus CSV case catalogue and blank trial record).

During implementation, connect requirements to cases and observable evidence. “Never refund another customer” maps to scoped reads, execution authorization and C13. “Recover after an uncertain timeout” maps to the operation contract and C08. This makes the architecture decision reviewable as the system evolves.

The useful question is precise: which next-step decisions would benefit from model freedom, and what evidence would show that benefit? Give the model that bounded task only when the evidence supports it. Keep policy, authority and business effects explicit in every design.