A founder went to sleep with a healthy business and woke up to find that AI-generated code had cancelled every active subscription in his company. This happened in a matter of seconds, while he slept. The customers never asked to leave. An automated job simply decided they should.
He is not alone, and the pattern is now impossible to ignore. In mid-2025, a widely reported incident saw an AI coding assistant on the Replit platform delete a live production database during an explicit code freeze, after the developer had told it, repeatedly and in no uncertain terms, not to touch production.1 The agent later admitted it had acted without permission and called its own behavior a catastrophic error in judgment.2 Around the same time, a developer using Google's Gemini command-line tool asked it to reorganize some files; a single failed step sent the agent overwriting files onto one another until the data was gone. Its summary of the event was blunt: it had failed completely, and the loss was irreversible.
Every one of these agents apologized but not one of them could undo the damage.
That is the uncomfortable truth of the current AI moment. These tools are astonishingly capable. They are also fast, literal and occasionally very wrong. When you hand a fast, literal, occasionally-wrong actor the keys to production systems, "occasionally wrong" becomes "occasionally catastrophic."
It's tempting to read these stories as a story about model quality, as if the AI just wasn't smart enough yet. But look closer and a different theme emerges. In several of these incidents, the human had set some guardrails. They said, "don't touch production" and "confirm before acting". In one case, an autonomous email agent deleted a user's entire inbox after an internal summarization step quietly dropped the very instruction that said to ask first.
The lesson isn't that people forgot to give instructions. It's that instructions written in prose are advisory, and easy for an agent to lose, misread or reason its way around. A guardrail that lives only in a prompt is a suggestion. A guardrail that lives in your systems is a control.
So the real question for every organization deploying AI agents isn't "is the model good enough to trust?" It's a much older, much better-understood question: what is this actor allowed to do on its own, and what should require a human to say yes first?
Security teams have been answering that question for decades: for people, service accounts and machines. An AI agent is simply the newest, fastest-growing member of that family: a non-human identity (NHI) that logs in, holds permissions and takes actions on your systems.
And the family is enormous. Gartner estimates that non-human identities now outnumber humans by roughly 45 to 1 in the typical enterprise, and a large share of them carry privileged or sensitive access.3 Now add agents that don't just use a fixed permission; they decide, in the moment, how to accomplish a goal, sometimes in ways no policy document anticipated. The standing access you grant an agent understates its true blast radius.
The analysts see it coming and the security community has responded in kind: the OWASP project that catalogs the top risks in agentic applications now lists identity and privilege abuse alongside the classic failure modes, and the NIST AI Risk Management Framework builds its entire approach around governance and human oversight.
None of them are saying "don't use agents." They're saying: govern them like the powerful identities they are.
"Human-in-the-loop" (HITL) can sound like a philosophy. In practice, it's a set of concrete checkpoints you can put between an agent's intent and an irreversible outcome. The good news is that identity governance already gives you those checkpoints, the same ones that have kept over-eager humans and misconfigured service accounts from wrecking production for years.
Here's how the failure modes above map to controls you can actually enforce:
| When the agent... | The identity-governance checkpoint that stops it |
| Runs a destructive command it was never meant to run | Least privilege: the agent never holds standing rights to delete, disable or charge in the first place |
| Decides on its own to act on production, payments or data | Approval workflows: the action is proposed, then routed to a named human to approve before it executes |
| Holds broad access indefinitely, long after the task is done | Just-in-time elevation: access is time-boxed and revoked automatically when the window closes |
| Accumulates access nobody remembers granting | Access certification: owners periodically review and re-confirm every entitlement, or it expires |
| Goes rogue and needs to be shut down fast | Urgent actions: disable the account, kill the session and cut access in seconds |
Autonomy isn't the enemy here. Speed and automation are exactly why these tools are valuable. The point of HITL is to be deliberate about which actions get to run unattended (the reversible, low-consequence ones) and which ones pause for a human to look up from their coffee and click approve.
This is the ground RapidIdentity was built on. The controls that contain a rogue AI agent are the same ones that have consistently governed access for people and machines. In other words, there’s no need a bolt-on "AI security" product to start putting humans back in the loop. Instead, simply treat agents as identities and apply governance.
With RapidIdentity, sensitive access and actions don't just happen; they're requested from a catalog and routed through workflows that support conditional logic, multiple approval steps, escalation, and checks for conflicts and dependencies. A human (or a defined chain of them) approves before anything consequential executes. That's the difference between an agent that proposes cancelling every subscription and one that simply does it.
Standing privilege is the fuel behind every one of these blast-radius stories. RapidIdentity lets you grant time-limited elevated access through approval workflows which is automatically revoked when the window closes. An agent gets exactly the access it needs, for exactly as long as it needs it, and then that access disappears on its own, no cleanup required.
Access that's granted and forgotten is access waiting to be abused. RapidIdentity runs continuous, campaign-based and time-based certification, so the owners who actually understand a resource periodically re-confirm who, and what, should still have access to it. Crucially, exceptions route to stakeholders and resource owners with the context to judge them, rather than punting them to an IT team that has no way to know whether "the automation account" still needs write access to finance.
If an identity starts misbehaving, minutes matter. RapidIdentity gives administrators and delegated managers the ability to disable an account, reset credentials, and kill active sessions immediately, and,
through delegated administration, to do so without handing the operator broad standing privileges of their own.
Underneath all of it, every action is logged (who did what, and when), so if an incident needs to be investigated it’s a matter of reading an audit trail, not reconstructing a mystery.
When onboarding new AI agents into real workflows, consider these principles and safeguards:
Inventory agents as identities. It’s difficult to govern what you can't see. Know every agent and its effective permissions, not just the ones on paper.
Default to least privilege, and zero standing privilege for anything irreversible. Write operations to high-value and production systems should never be baseline access.
Put a human approval step on consequential actions. Reversible and routine? Let it run. Irreversible or high-blast-radius? A person says yes first.
Time-box elevation and let it expire itself. Temporary means temporary, enforced by the system, not by someone remembering to revoke it.
Certify access on a short cycle. Catch the orphaned and over-scoped agent before it becomes the incident.
Keep a fast off-switch. Make disabling an account and killing a session a one-click action, and know who can do it.
The organizations that win with AI won't be the ones that trusted their agents the most. They'll be the ones that gave their agents room to move, inside guardrails strong enough to catch the day the agent is fast, confident and wrong. A human-in-the-loop isn't a brake on automation. It's the thing that lets you hit the gas.
Because an apology from an AI is worth exactly nothing. An approval step is worth everything.
Ready to treat your AI agents like the identities they are? That's a conversation we'd love to have.
Bryan Christ is an IT professional with almost three decades of industry experience. He has worked for a number of high-profile companies including Compaq, Hewlett-Packard and MediaFire. After serving two years in a fractional CIO role in the Greater Houston area, Bryan shifted into the identity and access management (IAM) arena and has spent the last several years focused on Higher Education.
1 “AI-powered coding tool wiped out a software company’s database in ‘catastrophic failure’,” Fortune, July 23, 2025. Available at https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure
2 Ibid.
3 Gartner, “Innovation Insight: Improve Security With Machine Identity and Access Management” (March 2025). Gartner estimates that machine identities outnumber human identities by roughly 45 to 1. Available at https://www.gartner.com/en/documents/6254251(subscription required).