Who is accountable when an agent acts without asking

A support agent books a refund, emails a customer and closes the ticket while the human owner is in another meeting. Nobody is sure who approved the threshold, or who can reverse the action if the refund was wrong.

NIST lists the risks that generative AI makes worse. One is how people and the system are set up to work together. People lean on the model too much, defer to it too fast and lose track of which calls the model made and which a person made. Another is third-party parts that arrive without a clear owner downstream.

Accountability before autonomy

An agent that can act needs an owner, like any other process. Before you widen what it may do, name the person accountable for that class of action, the limit that forces a pause and the record you'll keep when it acts. If those lines are blank, you've shipped a process with no owner.

NIST keeps bringing organisations back to four functions: govern, map, measure and manage. For an agent, that means listing where it's used, stating who may start it and planning how you'll switch it off when its behaviour drifts. A policy deck that never names a person beside each autonomous step won't help you on the day a customer complains.

Practical checks before you widen scope

1. List the actions the agent may take without a person signing off, and the actions it may only draft.
2. Put a named owner on each class of action, close enough to the work to spot a failure the same day.
3. Log the inputs, the action and the override path so a reviewer can reconstruct what happened.
4. Rehearse the shutoff. Make sure someone has actually practised stopping the agent before it runs on its own.

Why this matters once agents go live

Teams often celebrate the first unsupervised run. The harder moment is the first wrong one. If nobody owned the action, people argue about whose fault it was while the customer waits. Clear ownership turns that argument into a fix.

If you want a structured day on ownership and intake for AI use cases, the leadership day of our AI governance workshop is built for that conversation. A good session ends with a named person beside every action the agent can take.