The WAT Pattern: Workflows, Agents, Tools — And Why Most AI Systems Get It Backwards

The shape of every AI system that fails

The most common failure shape in production AI looks like this. The team builds an “agent” that takes a user request, reasons about what to do, and executes. The agent has a single big prompt and access to ten tools. In demos it’s magical. In week two of production it starts to drift — calling the wrong tool, misordering steps, hallucinating tool parameters, occasionally bricking customer data.

The team’s response is to lengthen the system prompt. The system prompt gets to 4,000 tokens. The drift gets worse, because now the agent is also trying to follow conflicting instructions. The team adds a “supervisor agent” that reviews each call. Latency doubles. Cost triples. The supervisor hallucinates approval for bad calls about 2% of the time. Eventually the team admits defeat and rewrites half the system as deterministic code.

This pattern is so common that I now expect it from any AI system designed in the last 24 months without an explicit architectural philosophy. The pattern has a name in our practice: agents-in-charge. It is the wrong place to put the probabilistic component.

The right pattern is WAT: Workflows, Agents, Tools. Each layer has a job and only one job.

What WAT actually is

WAT divides any AI system into three layers with clearly bounded responsibilities.

Workflows are the procedure

A workflow is a markdown document — or a checklist, or a state machine — that describes the procedure for accomplishing a task. It is declarative, deterministic, and human-readable. A workflow says things like:

1. Parse the inbound ticket.
2. Classify the topic.
3. If the topic is billing → route to billing tool.
4. If the topic is technical → search the knowledge base, draft a response, queue for human review.
5. Log the outcome to the audit table.

Workflows do not reason. They describe what a competent person would do, step by step. In our builds we author workflows as markdown files committed to the repo. They are reviewable, diffable, and the source of truth for “how the system is supposed to behave.”

Tools are the execution surface

A tool is a deterministic piece of code that does a single concrete thing: fetch a record, send an email, query an index, write to a table. Tools have strict input and output contracts. They are idempotent where possible, observable always, and fail loudly. They do not contain reasoning. They do not “decide” anything. They execute.

A well-designed tool is boring. It takes structured input, performs an action, returns structured output, and emits a log line. If a tool is making decisions, it’s a workflow in disguise and you’ve misclassified it.

Agents are the decision-maker

An agent reads the workflow, looks at the current state, and decides what tool to call next. That’s the entire job. The agent is the probabilistic component — it’s where the LLM lives — and it’s responsible for one specific kind of decision: which step of the workflow applies right now, and which tool serves it.

Agents in WAT are small. The system prompt is short. The toolset is curated to fit the workflow. The agent’s autonomy is bounded by the workflow it’s executing. When an agent doesn’t know what to do, it asks for clarification instead of guessing. When the workflow says “queue for human review,” the agent queues.

Why this changes the failure mode

In the agents-in-charge pattern, when something goes wrong, you don’t know whether the model misunderstood, the tool was broken, or the procedure was bad. Debugging is forensic archaeology. Fixes are heuristic — “add another sentence to the prompt and hope.”

In WAT, when something goes wrong, the failure has a clearly addressable layer:

• Workflow failure → the procedure was wrong. Edit the markdown. Re-run.
• Tool failure → the execution layer broke. Fix the code. Test deterministically.
• Agent failure → the model picked the wrong step. Investigate whether the workflow was ambiguous, whether the toolset is mis-curated, or whether the model genuinely can’t reason about this case.

Each layer has its own test surface. Workflows are reviewed by domain experts. Tools have unit tests. Agents have evaluation suites that measure “did it pick the right step for this state.” The failures don’t compound; they get caught in the layer that owns them.

The self-improvement loop WAT enables

The pattern unlocks an operating principle we’ve seen work consistently: every failure makes the system stronger.

When the agent picks the wrong tool, you learn one of three things:

1. The workflow is ambiguous → rewrite the workflow.
2. The toolset has a redundant or overlapping option → consolidate.
3. The agent genuinely can’t handle this case → escalate to human, log the case, expand the workflow.

In all three cases the update is a small, scoped, reviewable change. Not a prompt rewrite. Not a model migration. A diff to a markdown file or a single function.

Compare this to the agents-in-charge pattern, where every failure produces a “we need to update the prompt” Slack thread that lasts a week and produces no measurable improvement.

What WAT looks like in code

A minimal WAT system in production has this shape:

• workflows/ — markdown documents. One per business procedure.
• tools/ — deterministic functions. Strict typed I/O. One file per tool.
• agent.ts — the runtime loop. Loads a workflow, exposes the toolset, calls the model, dispatches to tools, logs every decision.
• evals/ — test cases that feed states into the agent and verify it picks the right next step.
• observability/ — every agent decision and every tool call emit a structured log. Cost meter and latency meter wrap every model call.

That’s the whole system. There’s no “supervisor agent.” There’s no “router agent of agents.” If you find yourself reaching for those, you’re papering over a workflow problem with more probabilistic complexity, which never works.

Where WAT fits next to MCP

Model Context Protocol is the wire format for tools. WAT is the architecture for how to use them. MCP makes tools portable across providers and clients. WAT makes the system maintainable across teams and quarters.

In our Custom AI Builds, every tool is an MCP server. Every workflow is a markdown doc. Every agent is a thin loop that reads the workflow and dispatches MCP calls. The pattern is boring on purpose. Boring scales. Clever doesn’t.

The summary

If you take one thing from this post: probabilistic models should make narrow decisions inside a deterministic procedure, not run the procedure themselves. Workflows are the procedure. Tools are the execution. Agents make the bounded decisions. Get that shape right and your system stays maintainable as it grows. Get it wrong and you end up with a 4,000-token system prompt that nobody can read and a 20% failure rate that nobody can debug.

---

WAT is the default architecture in our Custom AI Builds. If you’re staring at an agent that’s drifting and you want a second opinion on whether the architecture is the problem, schedule a call.