In our last issue I explained why it’s faster - all things being equal - to write new functionality from scratch versus updating an existing codebase with the same functionality. The reason? In sum: freedom of movement.

Today we’re going to look at why despite this advantage it’s often far costlier to rewrite code from scratch instead of updating existing code.
 

Consider Brasília

The short answer is that its riskier. This sounds absurd, does it not? Yet it is a documented phenomenon.

Interestingly for us, the reasons why we have analogues outside of software development, so I think it’d be nice to take a short diversion and take a look at cityscapes and urban planning.

Lets consider Brasília, Brazil’s inland capital.

Brasília[0] is a purpose built city, originally designed in 1956 to replace Rio de Janiero as the country’s capital. It was designed in a style the anthropologist James C. Scott describes as high modernism: an unjustifiably strong belief in the power of planning, top down control, steeped in scientism and an unwavering belief in technical progress.

It is, by all accounts, a beautiful city in its own right. It’s just not a very effective one.

Virtually all the needs of Brasília’s future residents were reflected in the plan. It is just that these needs were the same abstract, schematic needs that produced the formulas for Le Corbusier’s plans. Although it was surely a rational, healthy, rather egalitarian, state-created city, its plans made not the slightest concession to the desires, history, and practices of its residents.[1]

We might quibble with what it means for a city to be effective - and rightly so - but lets go with this definition: it allows people to move about, to mingle, to make use of space individually and together for commerce, recreation, shelter and ritual. Most of the patterns of use in a city arise from the bottom up, from people having the opportunity to interact with the space and with each other. 

These patterns develop to solve these things for the people living, working, and playing in the city. They just do not always appear legible to outsiders (a group which tends to include rulers and officials), Scott argues, and so are replaced by legible, orderly solutions. That is, solutions deemed legible and orderly to those implementing them. #
What in the heck does this have to do with software The comparison to software is imperfect. Software is a designed thing. It’s an abstract product of human thought which instructs a computer to manipulate bits.

Yet between people using the software to complete tasks - whether for work or play - and developers working in the source code over time, responding to the ways those users interact with the software, patterns emerge. Feature accrete, little workarounds are added to code, names in code and the world start to diverge. 

The code may work perfectly and yet remain illegible, demanding from an order-interested mind cleanup, reorganization, refactoring, or even slash-and-burn rewrite.

All carry risk - as does anything worth doing - but the rewrite is where things go wrong. Joel Spolsky wrote years ago about why Netscape 6 was so late in shipping[2]:

They did it by making the single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.

This isn’t some isolated temptation, it’s something software developers share:

We’re programmers. Programmers are, in their hearts, architects, and the first thing they want to do when they get to a site is to bulldoze the place flat and build something grand. We’re not excited by incremental renovation: tinkering, improving, planting flower beds.

To be perfectly clear this is a temptation shared not just by programmers but project managers, product owners, designers, pretty much anyone who has a hand in a product day to day - sometimes users, too!
 

The way forward for software

If you’re still thinking about Brasília, I have little to say about the solution for urban planning, at least with any kind of justifiable authority. 

However there are some answers I’m willing to provide for software like existing Django projects.

The first thing to do in the face of unnamed and dirty streets (confusing and ugly code) is to make a point of understanding it, labeling it, and cleaning it.

Sometimes instead of renaming things we can provide a map, first, of the existing names to something else they refer to. Rational renaming in software is good. Prematurely renaming, even rationally informed, can result in unintended confusion.

Cleaning should be an iterative process, performed ideally module-by-module and starting by tidying the smallest things first. There is non-trivial value in well formatted and consistently formatted code.

Unlike with cities and urban planning, you can achieve a state of orderly, beautiful software that would make a high modernist smile. However the path there should be taken step by step, not by leveling and rebuilding wholesale[3].

Why have you decided not to rewrite your Django project from scratch? Or have you instead decided that a full rewrite was warranted? Hit Reply and let me know.

Happy New Year,
Ben

[0] Readers note: I am judging the city based on second and third hand experience, I have not yet, at least, been to Brasília myself.
[1] Seeing Like a State, James C. Scott, p.125 (Yale Press, Amazon)
[2] Joel on Software: Things You Should Never Do, Part I
[3] Yes there _are _times when a total rewrite are warranted but for existing production software projects these are the exception, not the rule. If you’re a DHH fan and are thinking, “Well, Basecamp have done three rewrites now!” the difference is that each has been its own product, still separately supported, not an in-place replacement. 

More like this

• The module hierarchy and your Django project (This Old Pony #30)
• Did you know it's faster to rewrite code from scratch? (This Old Pony #28)