Why your existing observability stack misses everything
A standard web application observability stack — Datadog or equivalent, plus logs, plus error tracking — tells you when an HTTP request fails. It tells you when latency spikes. It tells you when an exception is thrown.
It tells you almost nothing about AI agent failures.
The reason is structural. Agent failures don’t look like exceptions. The most expensive failures are when the agent does exactly what its code asked it to, while making a bad decision. The HTTP call succeeded. The model returned a 200. The tool executed without error. The agent did the wrong thing at every step and your monitoring saw a green light through it all.
Building observability for agents requires a different stack. The traditional web-app stack is a necessary substrate; on top of it you need four agent-specific layers. What follows is the stack we ship by default.
Layer 1: Decision-level traces
Every agent action is the result of a decision. The decision needs to be the unit of observability, not the HTTP call that implemented it.
For every decision the agent makes — every “I will now call tool X with arguments Y” — emit a trace span that includes:
- The agent’s identity and tenant.
- The state the agent was in (current workflow step, prior actions in this session).
- The candidate options the agent considered (if available from the model’s reasoning).
- The decision made.
- The tool invoked, with redacted arguments.
- The outcome (success, failure, tool error).
- Cost and latency at the model layer.
These traces become the primary debugging surface when something goes wrong. The question “why did the agent do that” is answered by reading the trace, not by re-running the agent.
Most existing tracing libraries (OpenTelemetry, etc.) work fine as the substrate. The instrumentation is application-level. We typically use a thin wrapper around the model client that emits these spans automatically.
Layer 2: Cost and budget meters
Cost is the cheapest and fastest observability signal for agent health. A healthy agent has stable per-task cost. An unhealthy agent — looping, hallucinating, fetching too much — has cost that spikes.
The cost layer:
- Real-time aggregation by tenant, workflow, agent identity, model.
- Rolling baselines (28-day P95 per tenant, per workflow).
- Alerts when cost exceeds baseline by 3x in a 5-minute window.
- Hard ceilings that kill the agent at 8x baseline.
This is the same cost meter from the cost metering post. It’s worth restating because cost is so often the first symptom of agent trouble. Cost alerts often precede other monitoring signals by 30–60 minutes.
Layer 3: Output evaluation, continuous
The most distinctive layer of agent observability — and the one teams build last — is continuous output evaluation. The premise: agent quality drifts even when the code doesn’t change, because the model is upstream and changes underneath you, because the data changes, because user input shifts.
The standing eval pattern:
- A curated set of test cases that represent real workflow shapes (50–200 cases is usually plenty).
- Each case has expected outputs or quality criteria.
- The eval runs continuously (hourly, daily, or on every deploy).
- Results are tracked over time and alerted on regression.
Two evaluation modes both belong in the stack:
Exact / programmatic checks. “Did the agent call tool X?” “Did the response include source citations?” These are cheap and unambiguous.
LLM-graded checks. A separate LLM grades responses against criteria. More expensive, more flexible. Used for the “is this response actually helpful” question that programmatic checks can’t answer.
The combination catches both code regressions and model behavior drift. Without continuous eval, you find out about agent quality changes from customer complaints, which is the wrong feedback loop.
Layer 4: Tool reliability metrics
Tools are the agent’s actuators. Tool reliability is observable separately from agent reliability and worth tracking:
- Tool call success rate, by tool.
- Tool latency P50/P95/P99.
- Tool error rate, broken down by error type.
- Tool invocation pattern (which tools are called together, in what order).
The interesting metric here is tool-call patterns. When the pattern shifts — the agent suddenly calls Tool B 5x more often, or stops calling Tool C entirely — that’s often an early signal of agent drift. Watch the patterns even when no individual metric breaches a threshold.
The dashboard that matters
A useful agent dashboard has three sections:
The “is the agent healthy right now” section.
- Current request rate, cost rate, error rate.
- Active alerts.
- Recent decisions that took unusual paths (outliers).
The “is the agent drifting” section.
- Eval scores over the last 30 days.
- Cost-per-task over the last 30 days.
- Tool call pattern changes over the last 30 days.
The “what happened on incident X” section.
- Full trace replay for any agent session.
- Cost breakdown by step.
- Tool calls in chronological order with arguments and outcomes.
These three sections answer the three questions operators actually ask. Most agent dashboards we see answer none of them; they show generic application health metrics and call it observability.
The integrations worth building
Three integration points worth the engineering investment:
1. Connect cost data to revenue data. The CFO question — “are we making money on each customer” — requires joining cost-by-tenant with revenue-by-tenant. The join shouldn’t require a spreadsheet.
2. Connect incident data to the runbook. When an alert fires, the dashboard should link directly to the runbook section for that agent. The on-call engineer shouldn’t have to search.
3. Connect eval regressions to the deploy log. When eval scores drop, the dashboard should show which deploy preceded the regression. This catches “we shipped a prompt change and quality dropped” patterns in minutes instead of weeks.
What to defer
In agent observability there is real temptation to instrument everything. Some things are worth deferring until you have evidence they matter:
Per-token attention to model internals. Some teams instrument reasoning traces, attention patterns, intermediate model states. Interesting research. Not yet useful in production for most operators. Defer.
Vector embedding-similarity metrics. Tracking “embedding drift” of inputs over time sounds useful and usually isn’t. The signal/noise is poor. Defer.
Replaying historical sessions through new model versions. Sometimes useful, often expensive, mostly a research activity. Defer unless you have a specific reason.
The default stack above gives you enough to operate. Add the deferrable items if and when you find specific gaps.
What this looks like at small scale
For a single-agent, low-traffic system, the stack reduces to:
- Decision traces in a queryable log store (CloudWatch Logs, Datadog, or equivalent).
- Cost meter in a spreadsheet that updates daily (yes, really — at low scale this works).
- An eval suite that runs daily via cron.
- Tool reliability via the same trace data, aggregated in dashboards.
The full stack costs maybe a week of engineering at this scale. The point isn’t sophistication; it’s coverage of the failure modes that matter.
What this looks like at scale
At higher scale (multiple agents, dozens of tenants, real money on the table), the stack is closer to a small data product:
- Streaming traces to a real warehouse (BigQuery, Snowflake, or equivalent).
- Real-time cost dashboards with per-tenant attribution.
- Eval pipelines running every deploy plus continuous samples.
- Tool reliability with SLOs and on-call rotations.
- Connected to finance for unit economics.
The engineering investment is larger here — 1–2 engineers for a quarter — but the cost of not having it is the inability to scale the agent program safely.
The take
Agent observability is its own discipline. The web-app observability stack you already have is necessary substrate, not sufficient instrumentation. The four layers above — decision traces, cost, continuous eval, tool reliability — give you a system you can actually operate. Without them, you have an agent in production that you cannot debug, cannot improve confidently, and cannot defend when it misbehaves. With them, you have the foundation for everything else the AI program will do for the next several years.
Observability is a baseline deliverable in Custom AI Builds and a frequent finding in AI Readiness Audits. Schedule a call if you have agents running blind and want to talk about what to instrument first.
