AI Incident Response Works — So Why Are Engineers Losing the Skill?

AI incident response is getting good enough to handle the routine cases on its own — and that's precisely what makes it a problem. A recent article on the rise of 'AI SREs' makes an argument that deserves more attention than it's getting: the incidents automation quietly absorbs are the same incidents human responders used to learn from. The tool works, and its working is what erodes the skill that matters when it doesn't.

What AI SREs Actually Do Today

The current capability is real and worth stating plainly. AI tools can now inspect alerts, form hypotheses about what's failing, and implement fixes for routine incidents — the predictable, well-understood failure modes that make up most of an on-call engineer's page volume. For those cases, response times improve and humans stay asleep. That's a genuine win, and the source article doesn't dispute it.

The 1983 Paper That Predicted This Problem

The uncomfortable framing comes from an older source than any LLM. The 'Ironies of Automation,' a 1983 paper, described how automation leaves humans responsible for abnormal situations they have less practice handling — the more routine work a system takes over, the less prepared its human overseers are for the moment the system fails. The article grounds this in aviation: engines became so reliable there were fewer than one in-flight shutdown per 100,000 flight hours, and when things did go wrong, crews had as little as 117 seconds after the first warning to act. Rare failures demand practiced humans, and reliability itself removes the opportunities to practice.

Software operations is heading into the same structure. Routine incidents are how responders safely develop an intuition for how their systems behave and fail — the source article's own words. Remove them, and the engineers who must handle the severe, novel outage have watched automation work but haven't worked themselves. As the piece puts it: you might pick up a few things from watching Serena Williams play, but you only learn tennis by getting on the court.

Comprehension Debt Is the Risk Nobody's Pricing In

The article's sharpest concept is 'comprehension debt': as LLMs do more of the work, engineering teams accumulate a growing gap between how their systems work and how well responders understand them. It compounds like technical debt, but it's harder to see — there's no failing test for a team that no longer understands its own failure modes.

The proposed countermeasure is refreshingly unglamorous: realistic incident simulations and hands-on practice, run regularly — the article suggests roughly every six months — so human intuition stays current even as automation handles the day-to-day.

What This Actually Points To

My read: the real story here isn't that AI incident response is risky, it's that its costs are deferred and invisible while its benefits are immediate and measurable. A team adopting AI SREs will look better on every routine metric for quarters before the comprehension gap shows up — and it will show up on the worst day, not in a dashboard. That asymmetry is why 'comprehension debt' is the right name for it; nobody budgets for it because nothing triggers until it's expensive.

The practical implication is that simulation isn't a nice-to-have training budget line anymore — it's the replacement for the practice that automation removed. Teams that treat drills as redundant once AI handles routine pages are exactly the teams the 1983 paper warned about, forty years early. The honest position is that this is a manageable trade, but only if the practice is scheduled deliberately rather than assumed to happen naturally. It won't.