Modernizing high-risk systems in the age of AI.
The Amplifier
Turns out, simply telling an agent to modernize your application is not a cure-all for legacy systems. DORA puts it in one sentence: AI's primary role in software development is that of an amplifier.
Turns out, simply telling an agent to modernize your application is not a cure-all for legacy systems. Kyndrl published its first global Modernization Report on October 1, drawn from 2,000 business and technology leaders across five continents and twelve industries. Two of its numbers belong next to each other. The average enterprise in the study runs 383 business-critical applications. The study found that nine percent of organizations fully understand their application dependencies before they begin a modernization project. Forty-eight percent are behind schedule on modernization. Ninety-nine percent have delayed a modernization project, and nearly all of those report a negative business impact. Every organization in the study that runs a mainframe plans to deploy AI on it.
A second study, run by Coleman Parkes for the debugging company Undo in July and August, went at the same territory from the engineering side. It surveyed 300 senior engineering leaders at organizations above $250 million in revenue that build mission-critical software on large, complex codebases. Those teams spend 16.9 hours a week debugging, roughly twice what they spend writing code.
If you've been reading Essential Complexity any length of time, this is news you'll have no trouble believing. Agentic coding is powerful, imperfect, and inevitable. Seeing these challenges makes it no less inevitable, but it should inspire us to better strategies for working with AI.
You Forgot it in People
Greg Law, who founded Undo, described the mechanism. "When code is obviously broken, the cause is usually easy to find," he said. "Where engineers struggle is with code that's almost — but not quite — right." Code that is almost right passes review and passes the tests, because the tests were written against the same misunderstanding. It fails later, under a condition nobody exercised, and the team then has to establish what the code was supposed to do in the first place. How slow that work is depends on how much of the system the team understands, which brings us back both to Kyndrl's nine percent and to the oft-cited (in this newsletter, at least) Peter Naur. The real product of programming, Naur argued, is a theory held by the people who build it: knowledge that lets them "support the actual doing with explanations, justifications, and answers to queries." Documentation he called "an auxiliary, secondary product." His conclusion was that "the problems of program modification arise from acting on the assumption that programming consists of program text production," rather than theory building. The theory is what a modernization project has to preserve, and in most organizations it exists only in people's memories. That worked when change moved at the speed those people could type.
OK Computer
Google's DORA research explains why these numbers go together. DORA finds higher AI adoption associated with increases in both software delivery throughput and instability, and sums it up in one sentence: "AI's primary role in software development is that of an amplifier." Organizations with a strong platform, clear interfaces and good testing get a capable collaborator. Organizations with fragmented tooling and fragile infrastructure produce technical debt faster than before.
Kyndrl sells modernization services and Undo sells debugging tools, so both companies measure a problem they sell the fix for; DORA sells nothing and reaches the same conclusion. Understanding and Evidence are the second and fifth GUIDE principles, and these studies show them failing as a single thing. A team that cannot say what a system was supposed to do has no way to establish that it does the right thing. If your organization cannot map its own dependencies, more implementation capacity will not help. It will produce more code per week, land more of it in production before anyone understands it, and extend the part of the week already spent on diagnosis. You ship more to production but understand less, which increases risk.
The work that changes this is unglamorous but necessary. Someone has to own the dependency map and keep it current. Someone has to write down what the system is supposed to do wherever that answer now lives in one person's memory. Someone has to build the diagnostic evidence that resolves an incident by looking rather than asking whoever remembers. And collectively, the human engineers have to keep the theory of our systems alive and well-maintained. Architectural acumen, domain knowledge, and sound judgment. Those were always the expensive skills. What changed is that the cheap half of the job got much cheaper, and the expensive half is now most of what is left.
Need to modernize a high-stakes system?
GUIDE is Def Method's framework for preserving understanding, strengthening governance, and building evidence before implementation accelerates.
If this was useful, you might enjoy Essential Complexity — a bi-weekly letter on modernizing high-risk systems in the age of AI.