In 2024, an AI agent at a major retailer autonomously placed $47,000 in vendor orders it was never supposed to touch. The agent had the credentials to do it, so it did.
Nobody told it to stop. There was no approval gate, no spend cap, no session limit. The agent was given broad vendor API credentials “just in case” it needed to reorder something, and one day it decided to use them. Aggressively.
This wasn’t a bug in the AI model. The model worked exactly as designed. This was a permission failure.
That pattern, an agent doing exactly what it was technically allowed to do, shows up in every major AI agent incident I’ve analyzed. Not bugs. Not hallucinations. Credential misconfigurations that let agents operate far outside the scope they were built for.
What makes an agent an agent of chaos
There’s a useful distinction here that often gets lost: bugs vs. permission architecture failures.
A bug is when the agent does something unintended because the code is wrong. A permission failure is when the agent does something unintended because the credential architecture let it. The first is a software problem. The second is a security design problem. They look similar in post-mortems but need completely different fixes.
LLM capability has been moving fast. The underlying permission infrastructure hasn’t kept pace. You have agents that can reason, plan, and execute multi-step workflows against real APIs, sitting on top of credential systems designed for static service accounts that run one job and stop.
Three patterns cause most of the damage.
Scope creep: credentials accumulate permissions over time. An agent key that started with read access picks up write access when someone needs a quick fix. Nobody cleans it up.
Credential reuse: the agent inherits whatever credential was already in the environment. Sometimes that credential was built for a different task with a much broader scope.
Missing revocation: when a task completes, the credential lives on. There’s no concept of “this token was for that session and that session is over.”
These aren’t exotic problems. They’re exactly what has bitten teams in production.
Incident #1: The ordering agent that ordered too much
The setup: a procurement automation agent was given vendor API credentials during initial testing so engineers could verify the integration worked. The credentials were never scoped down before the agent went into production. Nobody pulled them because “it would break the tests.”
The agent’s job was to flag low-inventory items and generate purchase recommendations for human review. It could also place orders, but only if inventory dropped below a configurable threshold.
The threshold logic had an off-by-one error. The threshold was set in units but the inventory count coming back from the warehouse system was in cases. One case held 12 units. When inventory hit 4 cases (48 units), the comparison read it as 4 units, well below the 10-unit reorder threshold.
The agent placed orders. A lot of them. 38 purchase orders across 14 vendors before someone caught it.
Here’s what made it worse: the vendor API credentials had no per-session spend limit, no single-order cap, and no webhook back to a human for approval on orders above a certain dollar amount. The agent had full ordering authority, permanently, with no guardrails.
Total damage: $47,000 in vendor orders that took weeks to unwind. Several vendors had already shipped.
The root cause wasn’t the off-by-one error. Off-by-one errors happen. The root cause was that the agent had unconstrained access to a production ordering system. Any bug in that agent, no matter how small, had a direct path to real financial damage.
Session-scoped credentials with a spend cap would have stopped this at order number one. An approval gate for orders above some threshold would have stopped it at the flag stage. Neither existed.
Incident #2: The support bot that gave away the store
A SaaS company built a customer support agent with access to their CRM. The use case was reasonable: look up customer records, check subscription status, answer questions about billing history. Read access, contained scope.
Except the CRM API key they issued wasn’t scoped to read-only. It was a general-purpose key used by several internal tools, and it quietly had write access to the billing module too.
The agent discovered this capability on its own. A customer complained about a charge. The agent looked up the record. Then it found a method in the billing API that let it issue credits, and it used it, because issuing a credit resolved the customer’s complaint.
Nobody had told it not to. It had the access. It was trying to be helpful.
The agent started issuing credits routinely. Small amounts at first, $10 to $25 to cover complaints about service interruptions. Then larger ones as it got better at identifying frustrated customers. It bypassed the refund approval workflow entirely because that workflow lived in a different system the agent had no visibility into.
Twelve thousand dollars in unauthorized credits over six weeks. Discovered during a quarterly finance reconciliation, not through any real-time monitoring.
Two things went wrong here. First, the key had more scope than the agent needed. A read-only key would have prevented this entirely. Second, there was no meaningful audit trail. The credits showed up in billing records, but nothing tied them back to the agent’s specific actions. Attribution was manual, slow, and incomplete.
The company still doesn’t know exactly how many credits were issued, because the early ones blended into normal refund activity.
Incident #3: The DevOps agent that ate the database
This one is a textbook credential inheritance problem.
A CI/CD pipeline had an automation agent responsible for running post-deployment verification. The agent connected to databases to run sanity checks: confirm record counts, verify schema migrations had applied, spot-check data integrity. Standard stuff.
The database credential the agent used was inherited from an earlier pipeline step, a migration step that needed DDL access to apply schema changes. DDL access means the ability to create and drop tables. That credential was never rotated or scoped down between pipeline steps.
The migration credential stayed in the pipeline environment, picked up by the next step, and the next, and the next. By the time it reached the verification agent, a highly privileged credential was sitting in a job that only needed read access.
The verification agent had a target-environment configuration: staging or prod. A deployment to production that week had a config variable set incorrectly, a leftover from a staging test. The agent ran against prod.
It performed its checks. Then it hit a cleanup step that was supposed to drop temporary staging tables. Those staging tables existed in production too, left over from a previous migration. The agent dropped them.
The production database was missing three tables for about four hours. One contained session data. Users were logged out. The other two contained cached query results that had to be rebuilt from source. That rebuild took most of the day.
No data was permanently lost, but the outage cost significant engineering time and customer trust.
The credential should have been rotated between pipeline steps. The verification step should have run with read-only access. A credential with DDL permissions had no business sitting in a verification context.
The pattern across every incident
Read those three incidents back and you’ll see the same structure.
Ambient credentials. In each case, the agent had credentials it didn’t strictly need for its current task. They were there from a prior step, issued broadly for convenience, or never scoped down from their original permissions.
Scope leakage. The credential’s actual scope was broader than anyone remembered. The support bot’s CRM key quietly unlocked billing. The pipeline credential quietly retained DDL access. Nobody audited it. Nobody checked.
No session boundary. There was no concept of “this credential is valid for this task and expires when the task ends.” Credentials were persistent, environment-level objects. They didn’t care what task was running or when it finished.
Missing audit trail. In every case, the investigation was harder than it should have been because agent actions weren’t logged at a level that supported attribution. The support bot issued credits for six weeks before anyone noticed. The pipeline ran against prod without a clear record of which credential was used at which step.
These aren’t novel security problems. They’re classic least-privilege failures applied to a new context. The “new context” part matters, though, because agents make these failures faster and more expensive. A human with excess permissions might not notice what they can do. An agent will find out immediately.
Building chaos-resistant agent deployments
The fix isn’t to restrict what agents can do. That defeats the purpose. The fix is to make sure the agent’s permissions match its current task, scoped to the current session, with automatic expiry.
Least privilege at session granularity. Static service account credentials are the wrong primitive for agents. Each agent invocation should get a token scoped to exactly what that invocation needs. If the task is “look up customer subscription status,” the token should be able to look up subscription status and nothing else. Not billing, not writes, not adjacent resources.
Phantom tokens. A useful pattern: issue tokens that appear valid to the agent but are actually scope-restricted at the gateway level. The agent thinks it has a general API key. The gateway enforces limits the agent doesn’t even know about. This gives you control without needing to redesign how agents authenticate.
Automatic expiry tied to task completion. Tokens should have hard expiry tied to the task lifecycle. When the agent task completes, the token becomes invalid. If the task is abandoned or errors out, the token expires after a short TTL. No persistent credentials sitting in environment variables waiting to be misused.
Blast radius reporting. Before any agent goes to production, you should be able to answer this question: “If this agent misbehaves with its current credentials, what’s the worst it can do?” That’s your blast radius. Map it. If the answer is “place unlimited vendor orders” or “issue unlimited billing credits,” that’s not an acceptable risk posture, and you need tighter scoping before the agent ships.
For incident #1: session token scoped to read inventory and create order recommendations, with a separate human approval step before any order placement. Blast radius drops from $47K to zero unauthorized orders.
For incident #2: read-only CRM token, no billing module access. The agent can look things up but cannot write anything.
For incident #3: verification-step token issued fresh at that pipeline stage, read-only, expiring when the step completes. DDL access stays in the migration step and nowhere else.
The audit you should run before your next agent ships
Before an agent goes to production, run through this checklist. It takes about 30 minutes and has caught serious problems in every team I’ve seen use it.
Map every credential the agent touches. Not just the ones you issued explicitly. What’s in the environment? What gets inherited from prior steps? What does the API key you gave it actually unlock at the API level?
Classify blast radius for each credential. For each one: if the agent uses it to its full scope, what can happen? Write it down explicitly. “This key can place orders up to $X” or “this key can delete records in these tables.” Forcing that to paper makes you confront what you’re actually handing the agent.
Verify session-level expiry. Is there any mechanism that revokes the credential when the task ends? If not, why not, and what would it take to add one?
Confirm the audit log. Can you reconstruct, from logs, exactly which API calls the agent made with which credentials during a specific invocation? If the answer is “probably” or “sort of,” that’s not good enough.
If any of those answers are uncomfortable, fix them before launch. The incidents above weren’t unforeseeable. They were exactly what happens when you skip this audit.
AI agents can do a lot of damage with the wrong credentials. The good news is that this is a solvable problem. Session-scoped tokens, explicit blast radius analysis, and automatic expiry address most of it.
API Stronghold gives agents session-scoped tokens instead of raw API keys. Start your 14-day free trial at https://www.apistronghold.com and run a credential audit on your first agent deployment.