The reflex that produces bad systems
When teams encounter a new problem and decide AI is part of the solution, the default reach is to build an agent. The framing is “we’ll make an AI that does X” — where X is the desired outcome. The architecture follows: a system prompt, a toolset, a reasoning loop, a deployment.
Half the time this is the right architecture. Half the time the right answer is much simpler: build a tool. A tool is a deterministic function with optional AI inside it, that other systems can call. No reasoning loop, no system prompt, no agent identity.
The heuristic for which to build is straightforward. The heuristic is also not well-known, so teams over-reach for agents and end up with systems that are harder to operate than they need to be.
The heuristic, stated simply
Build a tool when:
- The task has a clear input shape and output shape.
- The task is the same every time.
- The “reasoning” is bounded and doesn’t require decisions about what to do next.
- The output is data, not action.
Build an agent when:
- The task requires deciding what to do based on context.
- The path through the task varies depending on what’s found along the way.
- Multiple steps with branching are required.
- The output is a sequence of actions taken on the world.
That’s the entire heuristic. Most teams know they should ask this question. Most teams don’t, and the systems show the cost.
Examples of each
A few examples to make the boundary clear.
Tools (no agent needed)
- Classify a support ticket into one of 8 categories. Clear input (ticket text). Clear output (category enum). No branching. This is a tool. A 50-line wrapper around a single model call, with a structured-output schema.
- Summarize a meeting transcript into 5 bullet points. Clear input. Clear output shape. Same every time. Tool.
- Extract structured fields from a PDF invoice. Clear input (PDF). Clear output (typed fields). Same every time. Tool.
- Generate a 280-character tweet from a longer text. Clear input, clear output, no decisions. Tool.
Agents (agent architecture warranted)
- Handle inbound customer support tickets end-to-end. The path depends on the ticket type. Some tickets get auto-resolved, some get drafted responses, some get escalated, some need additional info. Multi-step with branching. Agent.
- Research a sales prospect across multiple sources and produce a brief. The agent decides which sources to query, what to look for, when to stop. Agent.
- Drive a multi-step refactor across a codebase. Read code, decide what to change, change it, verify, possibly back out. Agent.
- Triage incoming product feedback and route to the right team. Multiple decisions, multiple branches, action-taking. Agent.
Why this matters
Building an agent for what should be a tool produces several specific costs.
Operational complexity. Agents have system prompts, reasoning loops, observability surfaces, kill switches. Tools have an input and an output. The operational surface of an agent is 5-10x that of a tool.
Cost. Agents typically make multiple model calls per task (planning + execution). Tools make one. For high-volume tasks, the cost difference is large.
Reliability. Tools fail in predictable ways. Agents fail in unpredictable ways. When something goes wrong with a tool, the diagnosis is “the input or the model call.” When something goes wrong with an agent, the diagnosis is “anywhere in the reasoning chain.”
Composability. Tools compose cleanly. Other systems can call them, including other agents. Agents compose poorly. Agent-of-agents architectures are fragile.
Cache-ability. Tool outputs can be cached by input hash. Agent decisions are non-deterministic and rarely cacheable. For repeat traffic, tools deliver enormous savings.
The cumulative effect: a system built with tools where tools suffice is dramatically cheaper to operate than a system built with agents everywhere. Often the cheaper system is also more reliable, because the simpler architecture has fewer failure modes.
When teams get this wrong
Three patterns of misjudgment we see:
1. Over-agent-ifying for prestige. Building agents sounds more impressive than building tools. Demos of agents are more exciting than demos of tools. Teams sometimes reach for the more complex architecture because it sells better internally. The system suffers.
2. Treating “AI did it” as agency. Some teams think any AI involvement requires an agent. This is wrong. A function that calls an LLM is still a function. It doesn’t need a reasoning loop, a system prompt about its identity, or a kill switch separate from the calling code.
3. Building an agent to compose tools that should themselves be tools. A common anti-pattern: an agent whose only job is to call three tools in sequence. The agent has a complex prompt explaining when to call which. The right architecture is a simple workflow that calls the tools directly — no agent needed.
The mixed pattern
In practice most real systems combine both. An agent runs the top-level workflow; the agent calls tools to do specific bounded work; the tools are simple.
This pattern is the WAT pattern in different terms: the agent is making decisions, the tools are executing. The discipline is to push as much as possible into tools and keep the agent layer thin.
Concretely, for a customer support system:
- Tools: classify ticket; summarize ticket history; fetch customer record; check order status; draft canned response in category X; send email; create internal note.
- Agent: the top-level loop that decides which tools to call based on the ticket’s characteristics.
The agent layer here is small — maybe 200 lines of code, with a system prompt that explains the workflow and provides the toolset. The tools do the actual work. The total system is dramatically simpler than an “AI-powered support agent” architecture that puts all the reasoning at the agent layer.
When to refactor toward tools
If you’re maintaining a system today that’s heavy on agent layers, indicators that some of it should become tools:
- A specific agent that always calls the same sequence of model calls. → Convert to a tool.
- An agent whose system prompt is mostly instructions that don’t involve decisions. → Convert the instructions to deterministic code; convert the model call to a tool.
- An agent layer that calls one other “sub-agent” that calls one model. → Collapse the two layers into a single tool.
- An agent that’s hard to test because its behavior is non-deterministic on the same input. → It’s probably a tool that was over-architected.
Refactoring agents to tools is usually a net reduction in code, in cost, and in failure modes. The refactor is small per change. The cumulative effect is meaningful.
The take
Build a tool, not an agent, whenever the heuristic allows. Tools are simpler, cheaper, more reliable, more composable. Agents are appropriate when decisions and branching are required, not as the default. The discipline of asking “is this a tool or an agent” at architectural decision points produces systems that scale. The lack of that discipline produces over-architected systems with agents everywhere, none of which are necessary.
Architecture review is part of the early phase of every Custom AI Build. If you’re staring at an agent that feels like it should be simpler, schedule a call.
