AI がインシデント対応を変革、しかし難問は依然として人間の領域

Artificial intelligence is rapidly changing how engineering teams respond to production incidents, offering the ability to summarize incident channels, analyze unfamiliar code, suggest remediation steps, generate pull requests, and increasingly assist with diagnosis. But according to a recent discussion highlighted by Uptime Labs during itsIncident Fest, the growing use of AI in incident response presents a paradox: the more routine work AI automates, the more important human expertise becomes when systems fail in ways that are novel, complex, or unexpected.

The discussion brings together perspectives from Uptime Labs, Chime, and Rootly to explore what happens when AI becomes another participant in the incident command center. The central argument is not that AI should be rejected, but that organizations need to be deliberate about how they introduce it. AI can remove significant cognitive load from responders, but teams must avoid allowing automation to erode the skills, situational awareness, and decision-making capabilities that are essential when AI itself reaches its limits.

The potential benefits are significant. Uptime Labs cites research discussed by J. Paul Reed suggesting that when AI diagnostic recommendations are correct, human users can perform substantially better than they do without AI assistance. At the same time, the same research highlights the danger of incorrect recommendations: misleading AI assistance can significantly degrade human performance compared with working without AI at all. The lesson is therefore not simply to "use AI" but to understand when its output can be trusted, how it should be challenged, and when humans need to take back control.

One of the most compelling ideas raised in the discussion is the Leftover Principle. As automation takes over routine tasks, the work left for humans increasingly consists of the unusual, ambiguous, and difficult problems that automation has not been able to solve.

Incident response is particularly vulnerable to this effect. If AI becomes increasingly capable of handling straightforward failures, engineers may encounter fewer routine incidents and therefore receive less practice in responding to them. When an exceptionally complex failure eventually occurs, the engineers responsible for managing it may have less hands-on experience than previous generations.

This creates what Uptime Labs describes as several interconnected risks: the remaining incidents become harder, human skills can atrophy through lack of practice, responders may lose situational context because they enter incidents only after AI has already performed much of the initial investigation, and organizations may create an accountability gap in which humans remain responsible for decisions without maintaining the expertise needed to make them confidently.

The concept echoes decades of research into the ironies of automation. The more organizations automate, the more they can inadvertently weaken the human capabilities needed when automation fails. For incident response, that means organizations cannot assume that AI will eliminate the need for experienced responders. Instead, they may need to invest more deliberately in maintaining those skills through game days, simulations, tabletop exercises, chaos engineering, and regular incident-response practice.

The National Institute of Standards and Technology (NIST) has also identified many of the same concerns in its2026 research into monitoring deployed AI systems. NIST specifically highlights insufficient research into human-AI feedback loops, the difficulty of scaling human-driven monitoring alongside rapid AI deployment, and the unresolved question of how automated monitoring should be balanced with human-validated monitoring. These findings reinforce the idea that simply placing a human somewhere in an AI-driven process is not sufficient; organizations need to understand how humans interact with AI recommendations and how those interactions affect decision quality over time.

There is another dimension to the challenge: AI-assisted development may dramatically increase the volume of software being created and changed. If teams can generate code, pull requests, and deployments at significantly higher rates, the number of changes entering production may also increase.

The Uptime Labs discussion frames incident frequency in simple terms: the number of incidents is influenced by both the volume of changes and the probability that any individual change introduces a failure. AI could increase the first variable substantially, while the quality of AI-generated code, tests, dependencies, and configurations will determine what happens to the second.

This makes established engineering practices more important, not less. Strong deployment controls, observability, feature flags, automated testing, resilience engineering, and rapid rollback mechanisms become critical safeguards in an environment where the pace of change is accelerating. The objective should not necessarily be to prevent every AI-generated mistake, but to ensure that mistakes are detected quickly, contained effectively, and reversed safely.

The most important takeaway from Uptime Labs' discussion is that AI could fundamentally change the role of the incident responder without eliminating the need for incident responders themselves. As AI handles more of the routine work, engineers may increasingly become responsible for the rare, ambiguous, high-consequence failures that remain beyond automated diagnosis.

About the Author

Craig Risi

Craig Risi is a man of many talents but has no sense of how to use them. He could be out changing the world but prefers to make software instead. He possesses a passion for software design, but more importantly software quality and designing systems in a technically diverse and constantly evolving tech world.

Craig is also the writer of the book, Quality By Design: Designing Quality Software Systems, and writes regular articles on his blog sites and various other tech sites around the world.

When not playing with software, he can often be found writing, designing board games, or running long distances for no apparent reason.

Show moreShow less