Understanding Technical Debt (and how to approach retiring it)

This originally ran on The Data & AI Ecosystem Substack

TL;DR:

• Tech debt is every tool, license, and legacy system that costs you time and money, and while it may do something important, it is not doing it efficiently, and nobody will bother replacing it due to the enormity of the work
• Tech debt has three different costs: the resources required to keep it working, the technology itself, and the opportunity cost/ operational efficiency you lose because the system doesn't evolve (this is the biggest cost)
• While AI is not a silver bullet, the capabilities it offers to work out what a legacy system actually does, recode/ rework it and aid in its transformation is a game changer and can reduce timelines and requirements of specialist expertise
• Proper tech debt transformation should be embedded in the existing operational evolution that organizations are already doing for AI; this should be done for the right systems (not everything fits) with the right type of resources

The world of technology is a confusing space now. There are technologies for everything, and very few organizations have a real grip on the different tools and software they're using.

This is extremely apparent in large organizations, where ineffective paid subscriptions and unused tools exist throughout.

And then there is legacy tech!

The joys and wonders of the technologies that were created before most people who are reading this were born. Or before all but one person on the team actually joined.

Realistically, it is important; it probably does something essential, and only one person knows how to keep it alive. And nobody in the organization has the guts to rip it out, and instead they turn a blind eye to this type of technology, continuing to pay for it whether it is a good investment or not.

Welcome to the world of Tech Debt!

Combined, tech debt encompasses all the tools, software and technologies that cost time and money to operate. It might solve one specific problem but creates a whole host of issues nobody wants to touch.

Having worked with a ton of these organizations, I would argue this is the single largest business problem in most companies. The problem might come from the plethora of technologies, how they are being used incorrectly, or the legacy tools that have stuck around (take your pick).

No matter the cause for it, organizations have turned a blind eye to it for way too long, and with AI, it is time to work to solve these issues.

So that's what we will do today in this article. It's the first of three articles on debt; next week we will tackle data debt and then the new category of AI debt (which is growing faster than either).

And just to be clear upfront, I am not arguing that you can eliminate all tech debt or that you even should. What I am pushing is that AI enables organizations to take a run at the highest value tech debt without drowning in resource suck, scope creep and unexplained breakages. This is a thought experiment built on an existing problem/definition, but that is what unique perspectives on the Data & AI industry should be, rather than generic AI-slop regurgitation.

What Tech Debt Actually Is (And Where It Lives)

The phrase "tech debt" is definitely overused. I would say it's probably being thrown around more than it should be, often because it so aptly describes a state of technical challenge that a lot of people find themselves in.

In short, technical debt means taking shortcuts. In software development, it usually takes the form of prioritizing speedy delivery over good code which creates problems down the road. More broadly, to the average person and within the technology space, tech debt covers everything from bugs to legacy code to missing documentation.

The term was coined by Ward Cunningham in 1992 in an experience report about a portfolio management system called WyCash. His framing was around the idea of shipping code before you fully understand the problem is like taking on debt. His line was "Every minute spent on not-quite-right code counts as interest on that debt."

Hence, the debt metaphor.

The idea was expanded on by Martin Fowler with the technical debt quadrant. Consultants and thought leaders love quadrants, and this one compares whether you knew you were taking a shortcut (deliberate vs. inadvertent) against whether it was worth it (prudent vs. reckless). I like this framing because people often view tech debt as an evil that somebody instilled before they came around for some stupid, no good reason. But also sometimes the debt is inadvertent and prudent, and the reason it became debt is time comes back to bite decades old code/ technology (with nobody bothering to maintain or update it).

I have seen these tools in a lot of organizations I have worked for:

• A $7bn logistics company became reliant on an old ERP that served as its single source for truth across the entire organization. In the end, it was a crutch to everything they did. ERP served its purpose way back when, but the company kept taking deliberate shortcuts to let it do more than it should have done, leading to a monstrosity of a system.
• A $4bn media company used an old tool to measure and record viewership of its shows. The tool served its purpose at the time, but it just kept living on, and it didn't work well for new types of viewership, like YouTube views or video on demand. By the time I worked with them, there was only one person who knew how to use the technology and keep it up to date. He was putting off his retirement because nobody else knew how to use it.

As I frame up the rest of the article, I want to be careful here; I'm practically minded, and I'm not one to recommend that you modernize everything all the time just because your tech is a few years old. For example, a twenty-year-old system that correctly processes something important every day is fine and should probably be left alone.

The problem is not that they (and tech debt by consequence) exist; the problem is that nobody has done the analysis required to know which ones should stay that way, and which ones should not.

And with AI, there is such an opportunity to identify and replace like there never was before!

If you are able to share that would be much appreciated! I keep this newsletter free and available for more people to learn, and you, as the reader, are my main sources of growth, so please consider a free share if you found this article helpful!

Why Tech Debt Exists

Most tech debt exists because the project is enormous and nobody has the guts to improve the status quo or put the resources, money, and time to fix it.

Especially in corporate settings, executives are measured by their ability to deliver quick wins. Often, with short tenures, there is no appetite to reconfigure the foundational technologies and systems that keep the organization functioning (hey if it works, why fix it right?). That is, unless it is already a strategic priority, like an ERP replacement or a third-party tool that being sunset.

If you think about what this kind of endeavour actually looks like, this rationale makes sense:

• You need someone to read code in whatever language the tool was built in (hello PHP or COBOL, which just hit its 67th year anniversary…)
• You need to trace every downstream consumer of both the data, the outputs and the technologies it feeds into (I guarantee you that lineage has not been mapped in an up-to-date way)
• You need to work out which of its behaviours are deliberate business rules and which are workflows that has been taken on with processes built in like band-aids
• Oh and you need resources, which are hard to come by…

Historically, this has been a multi-year engagement for most legacy technologies and you likely have to run parallel systems for a good portion of that (further increasing the cost).

Let's also not forget the obsession that companies have about value. Because nobody has done this type of work before, the modernization effort and ROI is hard to price out and the business case is usually pretty thin. It ends up being much easier to leave the cost in and just ignore it, which so many companies continue to do.

But when this catches up with a company, it can go public in the worst way. Remember the Southwest Airlines software crash in 2022. After a winter storm hit, a crew scheduling system called SkySolver couldn't handle the volume of changes, and they had to shut it off entirely, with schedulers doing the reassignments manually. Over nine days Southwest cancelled more than 16,700 flights and led to over $1 billion in losses (plus a $140 million fine), the worst operational meltdown in US airline history.

The best part is that leadership already knew about this problem; SkySolver was understood internally to be antiquated and unfit for a major disruption. But it was still running because replacing it was expensive (though probably not as expensive as the reputation risk, operational losses and regulatory fines).

The Costs of Outdated or Inefficient Technology

But I would argue, the costs of tech debt are becoming a lot more real, especially with the wave of innovative startups trying to disrupt every segment of the economy.

Legacy companies are up against smaller, more nimble start-ups that aren't burdened by legacy systems and don't have a conservative mindset about change. And they are starting to lose market share. Think about the banks in the UK losing to Monzo or Revolut. Or the US brokerages that had to eliminate trading commissions due to Robinhood's. While this is based on an innovative business model, part of that is enabled by a lack of legacy tech.

This brings me to a quick section on the costs of inefficient tech.

The first is resourcing. This is partly internal, in that you have people whose careers are focused on keeping a single system alive. And then the biggest part is external, where expertise is brought in via expensive consultants/ contractors, often at a premium because this expertise is hard to find.

Think expensive contractor day rates or specialist consultancy retainers just to maintain the status quo. It is literally adding no value, just helping an old technology function at the low expectations you have for it. For example, COBOL still operates about $3 trillion in daily commerce through its systems, and most operators are retiring. This means a lot of banks are outsourcing and Senior COBOL contractors can cost $100–300/ hour, which is a lot all things considered.

The second cost is the technology. Part of this cost includes the impossible resources that you have to source from above, which obviously gets more expensive the more obscure the skills required. The other tech costs include licenses, infrastructure, and additional support. Licenses are straightforward. Infrastructure sometimes means maintaining other technologies to ensure this legacy tech can run. For example, I've worked with a company that depends on a tax software add-on to their extremely old ERP system. Since they haven't been willing to sunset the tax software, they still run the ERP as well, even though it is overdue to move past.

The last part of technology costs is additional support; I'm referring to those clauses hidden in SLAs that state any change with the system or tool has to go through their over-priced consultants/ resources, no matter how minor. These add up, and companies sometimes just treat them as a cost of doing business, even though they shouldn't be in today's age (especially when vendors outsource these tasks and charge you 1000% of what it costs them to actually do).

Deloitte's CIO research put the share of technology budget spent sustaining existing operations at around 55%, with the rest split between enhancing what exists and building anything new. Meanwhile, Protiviti's global technology executive survey puts over 30% of IT budget and over 20% of IT staff time specifically on managing technical debt, and "nearly 70% of organizations view technical debt as having a high level of impact on their ability to innovate." In a market where technological innovation is crucial for survival, this is huge! Which leads to the next cost…

The third cost is the operational efficiency lost. And this is the biggest cost, which, of course, goes unreported (because who has the time to calculate this). The key issue is a lack of change to keep up with the times. Basically, because the system can't change, everything around it has to adapt.

I'll reference my two previous examples to drive home this point:

• For the logistics company, the reliance on the old ERP as the single source of truth for all of finances meant that the company had about 40 Financial & Operational Directors to manually run reports and ensure the data matched up to allow the business to function. The CFO told me he didn't want to pay $1m for a data platform (which is not what it costs but oh well), but he's happy to pay $4m in salaries to compensate for that lack of platform.
• The media company was moving to a business model that relied on video-on-demand as a key revenue source. By not updating their tech to measure that, they could not properly track advertisement views across their programs. They were losing thousands of dollars in advertising revenue per day because they didn't have the tools to sell the slots! The opportunity cost here was huge, and you will see that in more and more organizations.

These costs don't show up on budget lines or expense reports, but people know they exist. Unfortunately, those who deal with them are usually close to the work and therefore don't have the influence to enact change…

All these costs have been well known for a while, but as AI makes an entrance and companies are forcing themselves to evolve, technology, resource and opportunity costs are in the crossfire for Execs. While this is stressful for employees, it also presents an opportunity because for once someone may have the guts to finally address these costs listed above.

And I'm going to outline how below.

Rethinking Tech Modernization Efforts with AI

To be honest, this section is more of a thought experiment, built on my past practical experience and what I've seen possible with AI. I'm currently running through one of these processes right now, but it still will be a few months till we've truly modified the legacy workflows and sunset a couple of the technologies we've identified.

The core premise is not to treat tech debt as a one-off change program, but to embed it into your strategic operational evolution, which every company is going through right now because of AI.

An individual program means you have to find funding for it, build a business case, and it ends up siloed from everything else (people carry on with their work while the change team does stuff on the sidelines). My view is that as companies begin to invest in AI through Chief AI officers, AI engineers, and Forward-Deployed Engineers (FDEs), those resources should focus on retiring legacy technology as they redefine workflows and systems within the organization. This is exactly why Anthropic and OpenAI are investing in an FDE building consultant network with Ode and The Deployment Company (what a name…) — these companies are more than just tech AI professionals; they are people who know how to create change through technological evolution, using AI as the catalyst.

And any company with the right support should be able to do this too (without hiring Ode or other expensive consulting firms).

Below is how I would, and currently do, run this (also, some people may think this is too high-level or theoretical, but this is exactly the process McKinsey or a big consulting firm would charge $500k-$10+ million to do a tech transformation):

1. Prioritize by the Three Costs — People in the organization likely already know which technology costs them the most in terms of efficiency, resources, and opportunity. Ideally, as you start to remap and review the workflows, you can more specifically assess which technologies they use. Either way, prioritize by finding the business owner and relevant stakeholders for each of the identified tools/ technologies and get them to articulate the costs. Again, the more detail or quantification, the better, but if the answer is obvious, don't overdo it
2. Get AI to Explain What the System Does — Between interviews with the owner and stakeholders, you have half the picture. The second half, a good AI model can help you do. AI can read the code, review the tech specs, trace downstream consumers or sources, figure out the business rules, and couple it all with the business processes surfaced in stakeholder interviews. That used to be months of effort from a specialist that can now be done in a matter of weeks. This kind of comprehension is what makes legacy tech sunsetting more possible. By the way, this also tells you if your task is too big to chew off with the resources you have available…Share
3. Redesign the Workflows — Don't rebuild the same process on newer technology and call it transformation. The work in this step asks what the process should look like now, given what's available, and then works backward to what the underlying system actually needs to do. If you have them, surface some use cases you've built with business stakeholders concerning the existing tech. At this point, you should have researched different technical options to replace your legacy technology. Again, AI can be a huge help for this in matching it up to existing process flows, but you will need a business stakeholder and an architect to verify
4. Fill in the Gaps with the Cheapest Options — Based on your redesigned workflows, identify which tools and processes can deliver each step. Remember, your replacement solution might not be one unicorn tool that replaces everything you are ripping out (I think this mentality often stops companies from replacing legacy tech). Instead, look at the tools you have and see how many steps can be automated using those. For example, I have a client using a Microsoft ERP, but hasn't even started using the included Copilot or accounting layers in the tool (and instead pays for other technologies to fill gaps that could be addressed). If nothing exists internally, then look at buying something that covers it, but consider what other gaps/ workflows it can help fill. The last option should be a rebuild — a homegrown technology/ system that is bespoke and a lot of work to develop/ maintain. Out of this step you get a list of where each technology (existing, new, and potential) slots in
5. Agree on Tech Strategy & Roadmap — As you set out the future tech architecture based on the above steps you will have a clearer picture of how the technologies fit into the operations of your organization. This is the tech strategy you need, but most organizations don't have (for some reason). With this you should have a clear picture about the path to tech replacement. Which leaves us to…
6. Replace the Legacy Tech — I'll explain how to do this below based on the team roles and who does what. Again, this is high-level and I aim to share some more specific, practical examples in the future. But the biggest things to remember when it comes to sunsetting technologies is:Run in parallel through a full business cycle to ensure the new system can produce matching outputs. Note any maintenance/ ops issues during thisGet the history out of the old system before you switch it offActually cancel the contracts because I can't tell you how many companies are still paying for tools they haven't used for years… (aka tell procurement you turned the tool off)Make sure there is a transition plan for any owners of legacy tech. They have institutional knowledge, so they probably fit well on whatever new tech you will be running. Also, you don't want them to slow you down (even though usually they want you to sunset whatever legacy tech they own)Define what off means because a lot of organizations don't actually have the courage to switch these tools "off", so make sure there is agreement across teams on what that meansDon't rip out foundational technologies without knowing everything it points toDon't be overambitious. For example, I mention COBOL a lot, but taking that out of the transaction banking system is not a simple ask, and I don't want to play it off that this approach can do that. On the other hand, a lot of companies have recently rewritten critical infrastructure (or a portion of it) in Rust (e.g., Discord, Cloudflare, AWS, Microsoft, Dropbox, etc.), which is no small feat, but totally achievable and provides insane results on the backend. Even one of my newsletter partners/ sponsors Estuary just rewrote their engine in Rust, and it is delivering 8x higher throughput on a single worker across all data flows

Now, none of this happens side of desk. You need people with a mandate, which brings me to the squad.

The Legacy Tech Retirement Squad

To do this well, you need a dedicated squad of technically competent resources who have an organizational mandate to help eliminate tech debt aligned with the business's workflows and future transformation. The squad's goal/ output isn't a migration. It's a recommendation on how to rethink workflows considering which tools to keep, how to retire the most cost-ineffective technologies and factor in how to properly migrate to new tools/ AI- and data-embedded workflows.

To do that you need the right squad. Here are the six resources you need:

1. The Change Expert — Call this the project manager or transformation professional (it might also be a third-party consultant). This person runs the whole process, liaises with the executive team, gets input from across the business, and owns the timelines/ roadmap. They work with the business process and tech owners to prioritize the redesigned workflows and plan the tech debt retirements
2. The Business Process Owners — These people are consulted on how the process runs today, how it should run in the future, and what existing workarounds exist within the tech stack. This is where you also get buy-in to the transformational state and make sure they feel comfortable working with a new technology instead of the legacy version
3. The Future State Product Manager/ Forward-Deployed Engineer — The person responsible for redesigning the process, workflow or capability given the available tools, skills and technology. This is where you blend strategic/ operational thinking with product management skills and technical fluency to figure out what can replace what already exists. This is more about merging the different skillsets with an AI/ modern mindset, as the detailed technical capabilities exist elsewhere
4. The Legacy Tech Expert — The biggest problem in tech debt is most of the system's functionality and reason to exist lives in someone's head. This person therefore acts as the source of truth on what the system does versus what the business needs done. They also should be able to point out the parts of the process that have been impossible to replace in the past. This is a mix of subject-matter expertise and institutional knowledge.
5. An Enterprise Architect — Ideally, there is an enterprise architect who owns how the existing legacy tech is connected. Otherwise there is some other data architect who has mapped out the data architecture. This improves the transition from the existing tech and is likely necessary in the long run. Unfortunately this is also usually a gap in a lot of organizations
6. An Engineer — A technical resource to run the code comprehension work, manage the outputs, and validate that the tooling works against its identified use case. Given you are replacing an existing operational technology, this shouldn't be something you outsource to AI. And I would also consider putting a full-time engineering resource on this, rather than having them do it side of desk

The main goal of this team (or legacy tech retirement squad) is to figure out how AI models and AI-enabled tools can help redesign legacy tech that should be sunset for a new workflow or process. This is part technology updating, part handholding of the change process, and all about getting people to think differently about how they work.

Okay, there you have it: my attempt at providing a path forward for a problem that has plagued every organization for decades. Again, I want to reiterate that I am not an expert in the technical due diligence of this area, but I have seen significant progress in approaching this in today's new AI world (especially with a strategic, process-workflow-first perspective).

Next week, we tackle the next debt on my list — data debt — which most organizations also have. What is worse is that it is one of the things holding back many companies from using AI properly in today's environment. Well, I have a few solutions for you, so hope you turn up and read it! Otherwise, have a great weekend and see you next Sunday!

Thanks for the read! Comment below and share the newsletter if you think it's relevant! Feel free to also follow me on Substack, LinkedIn,, or reach out if you are looking for some top-notch freelance consulting input! See you amazing folks next week!

Originally published at The Data & AI Ecosystem. It goes out every Sunday, with a shorter mid-week piece that doesn't run anywhere else. Subscribe here.