Wellfire Interactive // Django consultants for your existing Django site

Two reasons why software gets complicated - even Django sites (This Old Pony #34)

Another professional developer friend recently published some of his thoughts on software complexity, and I wanted to share part of this and my own thoughts.
 

Since the dawn of time (before software, there was only darkness), there has been one constant: businesses want to build software cheaper and faster.

It is certainly an understandable and laudable goal – especially if you’ve spent any time around software developers. It is a goal that every engineer should support wholeheartedly, and we should always strive to create things as efficiently as possible, given the constraints of our situation.

However, the truth is we often don’t. It’s not intentional, but over time, we get waylaid by unforeseen complexities in building software and train ourselves to seek out edge cases, analysis gaps, all of the hidden repercussions that can result from a single bullet point of requirements.

We get enthralled by the maelstrom of complexity and the mental puzzle of engineering elegant solutions: Another layer of abstraction! DRY it up! Separate the concerns! Composition over inheritance! This too is understandable, but in the process, we often lose sight of the business problems being solved and forget that managing complexity is the  second most important  responsibility of software developers.

Justin goes on to lament some of the issues that he’s observed with regard to different technologies[0].

Here’s my take:
 

From my own perch, working on customers’ existing web apps, there are two things which I see lead to overly complex solutions:

  1. Separation between developer and business problem/customer
  2. A separation between responsibility for greenfield development and long term maintenance

The first isn’t so much about distance between the developer and the customer’s needs – what agile is intended to solve – but empathy with the root problem. The problem of getting overly engaged with the “mental puzzle of engineering elegant solutions” doesn’t go away but is at least ameliorated when developers share similar incentives as the customer, or at least empathize with *their* problem, instead of the developer’s own.

The second is more strictly related to incentives. Most of the fancy new frameworks – even some of the more mature ones – advertise their strength in *starting* projects. It’s easier to get things built, it’s easier to move quickly – and this is often at odds with what’s needed to sensibly and sanely maintain software well after launch. I’ve noticed more unjustified complexity in projects that were handed off by a dev team for initial build than in those built and maintained by the same team.

I’m not going to say these are *the* reasons for greater software complexity, but they’re two tractable reasons.

One of the biggest problems I’ve observed - in the sample set of customer projects we’ve worked on over the years - is a lack of skin in the game. This comes in different forms, and as a general principle influences more than just software development, but it shows up in both the brief points I replied with above. Developers, teams, and agencies that find themselves divorced from their customer’s business problems and/or from the need to maintain said software are more likely to product unnecessarily complex software.

For one, in any instance, a team’s discount rate for development comes into play. If you’re writing prototype software just to _get funding _your discount rate is very high and maintenance needs be damned. But if you’re writing software to support a long running project with an international team, or a SaaS with a steady growth target, you have a lower discount rate on technical development. That is, of course, if you’re tuned into those goals. Throw-it over the fence prototyping and an emphasis on big launches has probably done little to help web software complexity.

And there are other technical reasons for greater software complexity, some too that are perfectly valid. Sure, using frameworks like Django adds some complexity, insofar as it may contain a lot of complex code - especially in the ORM - and it would be less complex to keep the data management code in your own app. But the added complexity comes with significant benefits in terms of development speed and simplicity such that this level of complexity is [usually!] worthwhile.

To reiterate, this isn’t an exhaustive analysis of software complexity or its causes, but just enough for you to think over this week.

Simply yours,
Ben

[0] Software Complexity Is Killing Us https://www.simplethread.com/software-complexity-killing-us/

 

Learn from more articles like this how to make the most out of your existing Django site.