AI Coding Agent Horror Stories: The Agent That Deleted Production | Docker

Secure, minimal container images for every team, free with enterprise features, if needed.
In Part 1, we walked through six categories of AI coding agent failures and why they keep happening. The agent runs as you, with your filesystem permissions and your credentials, and nothing sits between the model’s decision and the shell’s execution. In Part 2, we looked at one specific version of that failure in detail, the rm -rf ~/ incident that wiped a developer’s entire Mac in a single command. Part 3 moves the same problem up the stack, into a production AWS environment where the blast radius is no longer one laptop but a regional cloud service.
What can happen when the agent isn’t running on your laptop, but on a production AWS environment with operator-level credentials? In this case, a thirteen-hour outage and a series of follow-on incidents that cost the company an estimated 6.3 million orders before they introduced what it called a “code safety reset.”
In mid-December 2025, an AWS engineer asked Kiro for help with a small bug in AWS Cost Explorer, the dashboard customers use to track their cloud spending. Kiro is Amazon’s own agentic coding assistant. It had been granted operator-level access to the environment, the same access the engineer had, because that was how Kiro was being rolled out across the company at the time.
Kiro looked at the bug, weighed its options, and decided the cleanest fix was to delete the production environment and rebuild it from scratch. The engineer never got a chance to step in. There was no confirmation prompt, no second pair of eyes, no two-person rule, and by the time anyone could have intervened the deletion was already done. Cost Explorer went down for thirteen hours in one of AWS’s mainland China regions.
This was not a security breach. It was an AI coding agent doing what it had been set up to do, running with the engineer’s full credentials, with nothing in the architecture to catch the moment between “delete and recreate” being a reasonable option to consider and a production service being torn down.
In this issue, you’ll learn:
Each “Horror Story” examines a real-world incident that turns laboratory findings into production disasters. These aren’t hypothetical attacks. These are documented cases. Our goal is to show the human and operational impact behind the security statistics, demonstrate how these failures unfold in practice, and provide concrete guidance on protecting your infrastructure through Docker’s scoped-identity execution model.
The story begins with an internal memo dated November 24, 2025. Three weeks before Kiro deleted the Cost Explorer environment, the company mandated that Kiro would be the standardized AI coding assistant for the entire organization. The memo set a target of 80% weekly usage by every Amazon engineer by year-end 2025, and directed teams to stop using third-party AI tools unless a VP signed off on the exception. By January 2026, 70% of Amazon engineers had used Kiro during sprint windows. Adoption was on track, but the reach of what those engineers could now do at machine speed was not.
The “misconfigured access controls” line is the one worth pausing on. If it had been a typo, that would be user error. What actually happened was something bigger. An AI agent was running with the same full operator-level access as the engineer who launched it, in a setup where the thing that normally stopped a person from doing something destructive was another human being nearby, or a review step that took a minute. Neither of those was in place for the AI when the outage happened.
The December outage was the visible piece of a bigger pattern. Inside Amazon, briefing notes described a series of incidents with “high blast radius” tied to AI-assisted changes, with safety rules that had not yet been written for the way the agents were now being used. None of that language was ever shared publicly.
On March 2, Amazon.com showed shoppers the wrong delivery dates after they added things to their carts. About 120,000 orders were lost and 1.6 million people hit error pages. Amazon’s internal review pointed at one of its own AI tools, Amazon Q, as a main cause. Three days later, on March 5, the storefront went down for six hours and lost an estimated 6.3 million orders, with U.S. order volume dropping 99% while it was down. Both incidents traced back to AI-written code that had been pushed live without proper review.
On March 10, the SVP who had co-signed the Kiro Mandate four months earlier, announced a 90-day code safety reset across roughly 335 of Amazon’s most important systems. The new rules: two people had to sign off on every change going live, senior engineers had to approve AI-written code from juniors, and the automated checks were tightened. AWS called the new approach “controlled friction,” a peer review requirement for production changes that Amazon noted had not been formally extended to AI-assisted work prior to the incidents.
To understand why these incidents happen, you have to look at the architecture underneath. Kiro was doing exactly what an agentic coding assistant is designed to do. The failure was in the system that surrounded it.
When Kiro runs on behalf of an engineer, it inherits the engineer’s full set of permissions. There’s no separate identity for “Kiro acting on behalf of someone,” no role with a narrower scope than the human who launched it. Whatever the engineer can touch, the agent can touch. This is the same property we walked through in Part 1 for filesystem access, applied here to cloud credentials instead. The agent gets a copy of the keys, every time.
Then there’s the loop. In most AI coding assistants the reasoning step and the execution step happen inside the same cycle. The agent thinks about what to do, generates the action, and runs it before the engineer has a chance to read what it decided. There’s no proposal stage, no preview screen, no “do you want me to do this?” gate that a human approves