Two patterns that look the same and behave differently
In early 2024, “tool calling” and “structured output” were nearly synonymous. Both meant “get the model to produce JSON conforming to a schema.” Frontier providers rolled them out as similar features and the production patterns hadn’t yet diverged.
In 2026 they’ve diverged. They serve different purposes, have different reliability characteristics, and picking the wrong one for your task is a routine source of brittle systems. This post is about how to pick.
What each one is now
Tool calling
The model decides whether and which tools to call, with what arguments, as part of its reasoning. The output is a structured request to invoke a named tool. The orchestrator dispatches the call, gets a result, and feeds the result back to the model for further reasoning.
Tool calling is agentic by nature. The model is making decisions. The model can call multiple tools in sequence, can decide not to call a tool, can call the same tool twice with different arguments. The reasoning happens around the calls.
Structured output
The model is constrained to produce a JSON object conforming to a schema. The model isn’t deciding what to do — it’s being told “produce this specific shape of data.” The output is the data, not a request for action.
Structured output is declarative by nature. The model is filling in a form. There’s no reasoning about what to do next. The pattern is “give me these specific fields, populated with model-derived content.”
When to use which
The decision is mechanical once you see the shape. Use tool calling when:
- The next action depends on what the model decides.
- Multiple steps, with the next step depending on the result of the previous one.
- The model might choose not to act.
- You want the model’s reasoning to be visible.
Use structured output when:
- You know the shape of the output you need.
- There is no “next step” — the output is the final result.
- The model’s job is to populate fields, not to decide actions.
- You need maximum reliability on schema conformance.
The reliability difference
Here’s the part most teams discover the hard way: structured output mode is meaningfully more reliable than tool calling for filling in known schemas.
The numbers, from our production work:
- Tool calling with a JSON-shaped argument schema: ~94–97% schema conformance for moderate schemas. Drops sharply on deeply nested schemas (8+ levels).
- Structured output mode: ~99–99.9% schema conformance, even for deeply nested schemas.
The reason is mechanical. Structured output mode constrains generation at the token level — the model literally cannot emit tokens that violate the schema. Tool calling generates JSON in a less-constrained way and validates after. The validation catches most issues but not all.
If your task is “fill in this form,” structured output is the right tool. Using tool calling for this is a slow, expensive way to get worse results.
The reasoning difference
Conversely, tool calling exposes the model’s reasoning in a way structured output doesn’t. When you need the model to think out loud before acting — which is most agent work — tool calling gives you that visibility.
Concretely:
- Tool calling pattern: model produces some reasoning text, then a tool call, then more reasoning. You can read the reasoning to understand why the model chose what it did.
- Structured output pattern: model produces the structured object. No reasoning is exposed. The model may have reasoned, but you don’t see it.
For agents that need to be debugged, supervised, or improved, tool calling’s exposed reasoning is essential. For pure data-shaping tasks, the reasoning is noise.
A worked example
Consider a customer support workflow with three concrete tasks:
Task A: Classify the incoming ticket into one of 8 categories.
Right answer: structured output. The schema is { category: enum[...], confidence: number }. There’s no decision to make beyond classification. Use the more reliable mode.
Task B: Decide what to do with the ticket — escalate to human, draft a response, request more info, close as not-an-issue.
Right answer: tool calling. The decision depends on the ticket content, and the next action varies. Each option is a tool. The model reasons about which to call.
Task C: Draft a response to the customer, with specific fields the response must include (greeting, problem acknowledgment, solution, sign-off).
Right answer: structured output. The schema is the response shape. There’s no decision to make about what to do; the model’s job is to produce the response.
A single workflow uses both patterns, deployed correctly per task. Many systems use only one, and the wrong one for half their tasks.
What goes wrong when you mix them up
The common failure modes:
Tool calling for pure data extraction. Slower, more expensive, less reliable than structured output. We see this often when teams started with tool calling and never re-evaluated. The fix is mechanical: convert pure-extraction calls to structured output. Quality and latency both improve.
Structured output where reasoning matters. The model produces a structured response that turns out to be wrong, and you have no idea why. The fix: switch to tool calling so the reasoning is visible, even if the per-call cost is slightly higher.
Forcing all tasks through one pattern for “consistency.” This is the wrong consistency. Use the pattern that fits the task. The cost of architectural consistency at the expense of task-fit is reliability you can’t get back.
The hybrid pattern
A pattern worth knowing: structured output as the final step of a tool-calling workflow.
The orchestrator agent uses tool calling to gather information, make decisions, and prepare the response. Once all the data is gathered, a final structured-output call formats the data into the precise schema the consumer needs.
This gets you the reasoning visibility of tool calling and the schema reliability of structured output. The cost is one extra model call. For workflows where the final output shape matters and the reasoning matters, this is the cleanest pattern.
Provider differences
The two patterns are implemented differently across providers, which affects portability:
- Anthropic Claude has solid tool calling, decent structured output. Tool calling is the more polished surface.
- OpenAI GPT has the most polished structured output mode (the response_format = json_schema pattern is best-in-class). Tool calling works well.
- Google Gemini has both, with some inconsistencies on complex schemas. Improving.
- Self-hosted models vary widely. Some implement both well via tool-calling-aware fine-tunes; others handle neither cleanly. Test on your specific stack.
For multi-provider routing, the choice of pattern affects portability. Pure tool calling is more portable than pure structured output (more providers handle it well). If you’re committed to one provider, optimize for whichever pattern that provider does best.
The take
Tool calling and structured output are not interchangeable in 2026. Tool calling is for decisions and reasoning. Structured output is for known shapes and maximum schema reliability. Most real workflows need both, deployed correctly per task. The cost of getting this wrong is small per call and significant at scale. The cost of getting it right is one decision per task, made once.
Pattern selection is part of the architecture review in our Custom AI Builds. If you have tool calling everywhere and want a second look, schedule a call.
