The kill switch nobody tested
Almost every AI pilot we audit has a kill switch in the documentation. Almost none of them have a kill switch that has been tested in the last 90 days. The two facts together mean the kill switch doesn’t exist; it’s a story the team tells itself about being safe.
We learned this the expensive way. The first autonomous agent we shipped to production had a documented kill switch. The on-call engineer flipped it during the first incident. Nothing happened. The agent kept running. It took us 90 minutes to figure out that the “kill” command was hitting a configuration that the agent loaded on startup but didn’t re-read at runtime. The agent had to be terminated by killing the container, which we hadn’t documented as a fallback.
Cost of the incident: roughly $4,000 in inference and one rough customer call. Cost of properly designing the kill switch up front: about two hours of engineering. The ratio is not unusual.
This post is the kill switch design pattern that has held up since. It’s mechanical, boring, testable, and small enough to fit into any pilot.
The four requirements
A kill switch is real if and only if it meets four requirements. Anything less is decoration.
1. Activatable in under five minutes by someone on-call
The clock starts when an incident is detected and ends when the agent stops. Under five minutes is the standard because anything longer means the incident expands while you’re trying to contain it. The on-call rotation must include someone who can actually flip the switch — not “the engineer who built it” who happens to be on vacation.
2. Effective immediately, with no in-flight work
The switch doesn’t queue. It doesn’t drain. It stops the agent and prevents new actions, period. Some teams build “graceful shutdown” kill switches that finish in-flight tasks before stopping. This is the wrong default. Graceful is what you want when the agent is healthy; kill is what you want when it isn’t.
3. Verifiable from outside the agent
After the switch is flipped, you should be able to verify the agent is stopped without trusting the agent’s own telemetry. A separate observability surface, a separate ping, a separate process check. The agent’s own logs are not sufficient — when an agent is misbehaving, the misbehavior may include lying about its own state.
4. Reversible deliberately
Flipping the switch off doesn’t restart the agent automatically. Restart requires a deliberate operator action, with a documented review of what happened before the restart. The wrong pattern is “the kill switch is also the restart switch” — that pattern produces incidents where the agent gets restarted before anyone understands why it was killed.
The three implementation layers
Real kill switches work in layers. Each layer is independently effective, so a failure of any single layer doesn’t disable the kill.
Layer 1: Application-level disable flag
A flag the agent reads at every decision point. When flipped, the agent stops taking actions and exits its loop. This is the fastest layer when working — sub-second to take effect.
This layer is necessary and insufficient. It depends on the agent actually checking the flag, which depends on the agent being healthy. When the agent is misbehaving, the application-level switch may not work.
Layer 2: Container/process termination
The orchestration layer (Kubernetes, AWS ECS, etc.) terminates the agent’s container. Effective in 10–30 seconds. Doesn’t depend on the agent cooperating. Works even if the agent is in an infinite loop.
This layer is the workhorse. Every kill switch should be implementable at this layer with one command.
Layer 3: Credential revocation
Revoke the credentials the agent uses to act. If the agent uses short-lived gateway-issued tokens, revoke at the gateway. If it uses long-lived credentials, rotate them. Effective in 1–5 minutes depending on token TTLs.
This layer is the belt-and-suspenders layer. Even if the agent process survives, it can no longer take actions on the world. This is critical for high-impact agents because it’s the layer that’s hardest to subvert.
The runbook shape
Every agent’s runbook should have a kill-switch section with this structure:
## Kill switch — {agent name}
### To stop the agent
1. Flip the app-level disable flag:
`gateway:set-flag agent:{name} disabled=true`
(Effective ~5s. Verify in observability dashboard.)
2. If the agent is still active after 60s, terminate the container:
`kubectl delete pod -l app={name}`
(Effective ~20s. Verify the pod is not restarting.)
3. Revoke credentials if the incident involves external impact:
`gateway:revoke-credential service={service} agent={name}`
(Effective ~5m due to token TTL.)
### Verification
- Observability dashboard shows "agent stopped" status.
- No new entries in the agent's audit log.
- (For credential revocation) Provider-side audit log shows revoked credentials.
### To restart
Requires written approval from {role}. Document the cause and the remediation in {incident-log path}. Then:
`gateway:set-flag agent:{name} disabled=false`
And redeploy the container if it was terminated.
The exact commands depend on your stack. The structure doesn’t.
The testing cadence
The most important property of a kill switch is that it’s been tested recently. The cadence we recommend:
- Quarterly. Run the kill switch on the agent in production (in a planned window, low-traffic time). Verify each layer works. Document the test result.
- After every architectural change to the agent. New tools, new credentials, new deployment pattern — re-test.
- As part of incident response training. Every on-call engineer flips the switch at least once before they’re on call.
Tests that aren’t on the calendar don’t happen. Put it on the calendar.
What goes wrong in practice
The five most common kill-switch failures we see, in roughly the order we see them:
1. The flag is read on startup, not at runtime. Agent loaded config once and never re-checked. Fix: explicit polling of the flag, or a push-based invalidation.
2. The container restart policy is “always.” Killing the container just restarts it. Fix: the disable flag should also prevent re-deployment, not just terminate the running process.
3. The on-call engineer doesn’t have access. The kill command requires permissions the on-call doesn’t have. Fix: pre-provision the access; rehearse the command.
4. The kill is fast but the impact is already done. Agent had already initiated an irreversible action before the switch was flipped. Fix: design the agent so its high-impact actions are reversible or require pre-commit approval.
5. The kill switch was renamed three months ago and the runbook wasn’t updated. Fix: the runbook test is part of the quarterly cadence. Both should fail or both should pass.
The cultural piece
The hardest part of kill-switch design isn’t the engineering. It’s the cultural permission to use it. Teams routinely hesitate to flip the kill on a misbehaving agent because:
- “It might be a false positive.”
- “If we kill it, we have to explain why.”
- “It will look bad on the metrics.”
This hesitation is what turns five-minute incidents into ninety-minute incidents. The fix is a clear escalation framework: when in doubt, kill. Apologize later. Document. Restart with a real review. The cost of a false-positive kill is small. The cost of a delayed real-positive kill is large.
Engineering leadership should explicitly endorse this posture. The on-call engineer should know: nobody gets blamed for killing an agent that turned out to be fine. Everyone gets reviewed when an agent that should have been killed wasn’t.
The take
Kill switches are not a compliance artifact. They are an operational control that determines how much damage an AI incident can do. Building one is small engineering work — usually a day or two — and the difference between an agent with a real kill switch and one without is the difference between “we had an incident, we contained it” and “we had an incident, we explained it to the board.”
Build the switch. Test the switch. Use the switch when you need to. That’s the whole job.
Kill-switch design is baseline on every Custom AI Build we ship, and is one of the items we audit in an AI Readiness Audit. Schedule a call if you have an agent in production and you’re not sure whether the kill switch is real.
