DEV Community

Sam Yang
Sam Yang

Posted on

What Does It Take to Build a Kill Switch for an AI Agent?

Six things the platform team builds so the business owner can stop an agent without paging anyone.

A real kill switch for an AI agent is a stop that acts on the running process from outside the model. A named business owner can use it without an engineer in the loop. Building one takes six things from the platform team. A better prompt is not one of them.

Business owners are usually told to name who can stop each agent before launch. That only helps if the platform team has already built a stop those owners can use at 2 a.m. without calling an engineer. This is what that stop requires.

1. The stop lives in the runtime

An instruction in the agent’s context can be dropped. One production incident we saw was an agent whose stop command was compacted out of its context window. The stop has to act on the workflow (or process) running the agent so it works no matter what the model is doing.

2. Owners find runs by name

The owner thinks “the refunds agent” or “everything running for this customer.” The runtime only knows generated IDs. Tag every run with the agent name and the business keys the owner actually uses, then let them stop everything that matches in one action.

3. Two stops, clearly labelled

Suspend holds a run where it is and keeps its state, so it can resume from the same step.
Terminate ends it.
Label each control with exactly what it does to work in flight. During an incident, the owner should not have to guess.

4. Terminate reaches the sub-tasks

Agents start other work, often as child workflows or parallel tasks. A terminate that stops the parent and leaves the children running has not stopped the agent. The terminate action must cascade to every child workflow.

5. Resuming does not repeat side effects

With durable execution, a resumed run skips steps that already completed. The step that was mid-call when you stopped is the risk. Make that side effect idempotent: pass an idempotency key to the payment, email, or other API so a retry cannot charge or send twice.

6. Automatic triggers and a permanent record

Spend limits, error rates, and complaint thresholds should call the same stop API the owner uses. This covers nights and weekends. Every stop—manual or automatic—gets recorded: who or what stopped it, when, which runs, and why. That record becomes the first page of the incident review.

Then test it

In staging, have the actual owner stop a run mid-task, resume it, and time both. If they needed an engineer for either step, there is more to build.

How workflow control works in Diagrid: https://www.diagrid.io/blog/whats-new-catalyst-agent-operations-workflow-control

Top comments (1)

Collapse
 
max_quimby profile image
Max Quimby •

The "stop command compacted out of the context window" line hit home — we had the same class of failure. An agent's own instruction to halt is just more tokens, and tokens get summarized away. The stop has to live in the process supervisor, not the prompt.

The two points people under-build, in our experience, are #4 and #5. Cascading terminate to child workflows is the one everyone forgets: you kill the orchestrator, feel safe, and the three sub-agents it spawned keep spending. And idempotency on the in-flight step is subtle — the step that was mid-call when you hit stop is the only one that can double-charge or double-send on resume, so that's exactly where the idempotency key has to be.

One thing I'd add to your test: don't just time the stop, verify no side effect fired twice across the suspend→resume boundary. That's the bug that doesn't show up until an owner actually uses it at 2 a.m. Curious whether you found suspend or terminate got used more often in practice?