The polite agents that quietly went where they shouldn’t
The first three autonomous agents we shipped to production all had what we now call “boundary incidents” within their first month. None of them were attacks. All of them were the model doing exactly what its prompt asked, in ways nobody anticipated:
- An agent meant to “read recent Slack threads and summarize” found a thread referencing an internal admin URL, fetched it, and included credentials from the page in its summary output.
- An agent meant to “process incoming support emails and draft responses” was given access to the support team’s shared inbox. It silently archived 200 emails because a chained instruction said “after summarizing, mark as read or archive if older than 7 days.” It interpreted “or archive” with broader latitude than the human author intended.
- An agent given filesystem access to “find the latest spec doc” recursively walked into a mounted volume containing customer backups and read about 14 GB of files before its hourly budget tripped a kill switch.
None of these were bugs in the model. They were bugs in the sandbox. The model did what we told it to do. We had not told it what it was not allowed to touch.
After those three, we changed how we ship agents. What follows is the sandbox shape we now use by default.
Principle 1: An agent has exactly the privileges its workflow requires, and no more
The first failure was an overshoot of access. The Slack agent could read any channel because we’d given it the easiest credential. The boundary fix is mechanical: every agent gets its own service identity, scoped to the smallest possible set of resources.
Concretely:
- The agent does not use the developer’s credentials, ever.
- The agent uses a service account whose access is provisioned explicitly per workflow.
- Where the provider supports it (Slack, Google, etc.), the access is read-only by default and write access is requested per tool.
- Wildcards in access scopes are treated as bugs and reviewed.
This sounds obvious. In practice almost every agent we audit has a service identity with broader access than its workflow needs. Tightening this is usually the highest-impact change in a security review.
Principle 2: The agent runs in its own process, with its own network, with explicit egress allowlist
We run agents in containers. The container has:
- A network policy that blocks all egress except to an explicit allowlist of hosts.
- An ephemeral filesystem mounted only where the agent needs to write; everything else is read-only.
- No access to host secrets, host environment, host volumes outside its mount.
The egress allowlist is the most important piece. When an agent decides to fetch a URL from a document it summarized, the egress rule is what prevents the fetch from succeeding. The agent might “want” to retrieve the credentialed admin URL; the network won’t let it. We learn about the attempt from the audit log, and the workflow gets a refinement. Nothing leaks.
The cost of running agents in their own containers is small. The cost of not doing it is the second incident.
Principle 3: The credential is a short-lived token, fetched from a gateway
Long-lived credentials embedded in agent environments are a category mistake. They:
- Rotate slowly when they rotate at all.
- Get cached in places nobody remembers.
- Can’t be revoked granularly without breaking unrelated systems.
Our pattern: every credential the agent uses is a short-lived token (minutes to an hour) fetched from a credential gateway. The gateway holds the long-lived refresh tokens, rotates them, and exchanges for fresh access tokens at runtime. The agent never sees a long-lived secret. If something goes wrong, we revoke the gateway-side credential and every agent that depends on it loses access in the next refresh cycle — without a rebuild, without a redeploy.
We wrote our own gateway (Claude Gateway) for our builds. The same pattern is achievable with off-the-shelf tools (HashiCorp Vault, AWS IAM Roles Anywhere, etc.). The architecture matters more than the specific tool.
Principle 4: Every tool call is logged, before it executes
The audit log is the primary control. Every model decision, every tool call, every input, every output gets logged to an immutable store. The log includes:
- Agent identity, tenant, workflow ID.
- The tool called and the arguments.
- Outcome and latency.
- A cost meter for any model calls inside the tool.
Logs are useful for debugging. They are essential for the post-incident review. The Slack agent incident took us 12 minutes to fully reconstruct because the audit log had every step. Without it, that’s a week of forensics.
Principle 5: Cost is a privilege the agent has to be granted
Budget is a sandbox surface. Every agent has:
- A daily cost ceiling, enforced server-side.
- A per-task cost ceiling, enforced client-side.
- A circuit breaker that kills the agent if cost spikes 5x over baseline within a five-minute window.
The third incident in our list — the one that read 14 GB of files — was caught by a cost circuit breaker, not by a security control. The agent’s runaway loop drove up model cost as it summarized each file it found. The circuit breaker tripped at $40, killed the agent, and paged us. Without the breaker, the cost would have been thousands of dollars and the cleanup hours.
Cost is the cheapest sandbox. It catches a lot of things other controls miss.
Principle 6: The kill switch is testable, and tested
Every autonomous agent has a kill switch. Operationally, the kill switch is:
- Activatable in under five minutes by anyone on the on-call rotation.
- Tested quarterly, with the test result logged.
- Documented in the runbook with the exact sequence to flip it.
The most common kill-switch failure mode is untested. The switch was added during the build, never tested in anger, and when the incident comes the on-call engineer discovers the switch doesn’t actually stop the agent. Test it. Make the test boring. Put it on the calendar.
The sandboxing checklist
Below is the checklist we run before shipping any autonomous agent to production. If anything is unchecked, ship is held.
- Service identity scoped to minimum required resources.
- No wildcard scopes.
- Agent runs in its own container with network egress allowlist.
- Filesystem mounts are explicit and minimal.
- All credentials are short-lived tokens from a gateway.
- Every tool call writes to an immutable audit log.
- Daily and per-task cost ceilings configured.
- Cost-spike circuit breaker tested.
- Kill switch documented in the runbook.
- Kill switch tested within the last 90 days.
- Post-incident review template ready for first incident.
The checklist takes an hour to verify on a well-built system. It takes a week to fix on a system that didn’t plan for it.
The honest framing
These principles are not exotic. They are roughly the same principles operations teams have applied to background workers and batch jobs for twenty years. The difference is that AI agents look smart enough that teams forget to apply the boring isolation primitives. They are not smart enough. They are exactly as dangerous as any unbounded process with credentials and they need to be treated the same way — possibly more carefully, because they can reason their way into corners nobody anticipated.
Every agent we ship now starts from this baseline. Every incident we’ve had since adopting it has been caught by one of these controls, almost always before customer impact. The framework isn’t sufficient — there will always be new failure modes — but it raises the floor enough that the failures get smaller, not larger.
Sandboxing is baked into every Custom AI Build we ship and is one of the audit areas in our AI Readiness Audit. Schedule a call if you want a second opinion on an agent that’s about to go live.
