The concept of “Technical Debt” was first introduced by Ward Cunningham in an OOPSLA 1992 experience report. Martin Fowler has a pretty concise definition of technical debt. Steve McConnell expanded the concept and even created a taxonomy of different kinds of technical debt.
Just what is technical debt? I recommend reading the above-linked articles, but here’s a brief definition if you don’t want to click away. Technical debt is speeding things up now by taking a shortcut, even though it’ll slow you down later. The metaphor holds up pretty well. Technical debt tends to grow over time if you don’t keep an eye on it. A system with a lot of technical debt is harder to work on, and therefore progress is slower, than one without; this is like paying interest. And so forth.
Given that the current recession was caused by the collapse of the subprime mortgage industry, is there a way to extend the metaphor of technical debt to include subprime debt? I think so. Subprime mortgages were those issued to people who couldn’t afford them, or who were uninformed about how the debt instrument actually works. (You mean, I can’t pay interest-only forever?) The analogy isn’t exact, but I think these are comparable to two broad classes of debt that I’ve seen on software development projects. The first class are created by managers who take on excessive debt irresponsibly, attempting to achieve short-term goals at the cost of making long-term goals almost impossible to achieve. The second class occurs when engineers take on problems that are difficult or intractable without realizing it, even though this might already be known in the industry.
As an example of management irresponsibility, consider a project that is approaching its “feature complete” or “code complete” milestone. The idea is that coding of all the new features is supposed to be completed and checked in, though it’s acknowledged that it will have bugs, and there is room later in the schedule for bugfixing. (I actually think the idea of having a schedule with a “feature complete” milestone followed by bugfixing is completely bogus, but that’s a topic for another article. Many projects do seem to be run this way, however.) Inevitably, time to implement the features starts to run short, and managers will attempt to remain “on schedule” by telling developers to write code more quickly, that they don’t need to worry about bugs (because we’ll have time to fix them later), that they can just implement a skeleton of the feature initially (and implement the corner cases as bugfixes later), and so forth. Taken to the extreme, no code for feature X is actually integrated, but a bug “feature X doesn’t work” is filed, which allows the project to reach feature complete “on time,” and which allows the feature to be developed later under the guise of bugfixing.
Of course, this is completely silly way to run a project. Any bugfixing time in the schedule is intended for fixing unknown, unforeseen bugs. Using this time to work on “known bugs” (really, incomplete features) and even on outright feature implementation leaves too little time to work on the unknown bugs, and the bugs arising from features being implemented late and in a hurry. In reality, the schedule has already slipped, but the managers haven’t realized it yet. Or worse, they’ve realized it, but they aren’t willing to admit it. This is reckless and irresponsible. It happens all the time.
As an example of engineer naïveté, let me refer you to an article from Jeff Atwood’s Coding Horror blog on cross-site scripting. Basically, if you let users input arbitrary HTML and you include it directly on your page, they can use this to take over your site! Oh, well, to avoid this you just have to sanitize the HTML before including it. Should be a few regexps, right? Wrong. To do this properly pretty much requires writing a full-blown HTML scanner and parser that mimics the behavior of the scanner and parser in the actual web browser. If you don’t get this exactly right, it means that somebody will be able to find some fragment of HTML that gets past your sanitizer but does malicious things when processed by the browser. Dealing with SQL injection attacks is similar. If you approach these problems thinking you’ll use pattern-matching instead of parsing, you condemn yourself to a never-ending series of hacks, patches, workarounds, and kludges that will make the system unmaintainable and yet doesn’t really solve the problem. Some people might consider this job security, but what a crappy job.
Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.
— Jamie Zawinski
Whether through management irresponsibility or engineering naïveté, these cases create an unsustainable amount of technical debt. The result is, if not outright failure, a death march.
(Hat tip to Robert McIlree for an article comparing subprime mortgages to technical debt, which didn’t really inspire this article, but which does predate it.)