If you celebrated Christmas yesterday Merry Christmas! And if not, play with a computer in celebration of Charles Babbage’s 227th birthday.

Okay, back to Django.

If I had to synthesize what I’ve overheard - and read - people say about Django in 2018, I’d distill it down to two points:

1. Django is now the go-to for building a SaaS web application.
2. Django’s a relic of the past that’s been supplanted by newer technologies. And here’s the thing - in a way they’re both true.
    

   History and a theory of market share

   For a long while, Django had the feeling of being of two big web MV[C] frameworks, the other being Ruby on Rails. This is more about mindshare than anything, of course, as many enterprises were quietly running Struts and Spring and Whatever.NET. With frameworks like Django (and Rails) it seemed like people could build anything and everything - major newspaper sites, Instagram, Twitter (Rails), Disqus, and the error logging service Sentry…

Fast forward and Twitter long ago rewrote much of their platform on the JVM, Disqus has started using Go, and Sentry is now using Rust….[0]

It’s pretty obvious that Django (and Rails) are now dead, amirite?

Well, no.

The thing is, when Django (and Rails) came on the scene, the existing tools kinda sucked, certainly by comparison. You had Java, and you had PHP, and CGI servers - not exclusively of course! - tools and languages that didn’t lend themselves to developer love or a focus on the specific business problem. And then you have these frameworks in languages that appeal to people (probably different people, respectively), one focused on developer aesthetics and the other perfectionists with deadlines. In an era of dumb phones and slowish Internet connections there wasn’t much you couldn’t do in Django (or Rails) that you could do in another framework or language. Sure, you’d likely see performance deltas, but for most sites it wouldn’t matter, and where it did caching would fill that gap.

But a lot has changed since then, on both the demand side and the supply side.

On the demand side, we’ve seen an explosion of both people and devices accessing websites, over generally faster Internet speeds, with more data exchange both up and down. In terms of data exchange, Web 2.0 was a joke compared to today! I remember a training we used to give on my old team for senior military officers about Web 2.0 and the empowerment of regular “users” to publish for everyone else. But we were talking blog posts and wikis, not selfie videos and real-time connected apps.

So is the problem that Django (and Rails) can’t handle data uploads? Or polling? Or even real time connectivity?

Not at all! And this is where I want to talk about market share.

Usually when people talk about market share they use pie charts as their visual examples. I hate pie charts, but here we go.

What a pie chart lover would say in this instance, is that tools like Django (and Rails) have lost some market share because the pie has gotten larger (their pieces have gotten bigger even as the pie has gotten larger, but the fraction of the pie has gotten smaller). Sure, maybe, but whether that’s true or not it’s missing the point like the Buffalo Bills in the Super Bowl[1]. The dimensions of a pie chart are limited. They suggest a commonality and it’d be like comparing the “market share” of software libraries of Django and Cocoa. Now, that may well hint at something, e.g. the relative rise of web applications and a dip in the popularity of Mac desktop programming, but it’s not really comparing the same thing.

And what’s happened in the past 15, 10, 5 years is the dimensions have expanded. I’m mainly thinking supply side (the technology) but this is of course true on the demand side.

Instead of a pie, let’s imagine a square of tiny cubes, and instead of portion of area think portion of volume. If you add a dimension then you get a [bigger] cube, or at least a cuboid. Now it’s possible that the volume - the total market - could grow in different dimensions, or the shape could in fact change.

Before getting into weird geometry territory, it suffices to say that different needs and capabilities arise that are outside our favorite framework’s wheelhouse. If you wanted to write a globally distributed real time chat application I’d reckon you could do that with Django Channels, but something like Elixir/Erlang might be better suited to that. And if you have to immediately process large volumes of data then a fast compiled language like Go is probably the tool for your job.

Django (and Rails) have improved a lot over the years and the number of use cases they fill has grown. However the demand for specialty jobs has grown too, and the capability of specialty tools, too. Simply, the shape of the “market share” of web programming tools has changed.

But the thing is, while there’s been this growth of adopted use cases that go beyond what Django is quite good at, and some of these tools have proven themselves to be competent at the job of a core web framework like Django, there’s a HUGE area, er, volume, of problems for which these other tools aren’t as good as Django (yeah, or even Rails). The area, sorry, volume of share of the problem space where Django just sucks as a solution is far smaller than you might think from reading comments on SV-centric sites like Hacker News.
 

A corollary in databases

I remember when MongoDB was the hottest thing anyone was talking about. CouchDB then MongoDB, document databases in general - which were not actually new, by the way - these where going to totally supplant relational databases like PostgreSQL and MySQL. You could have been forgiven for thinking that. Feature development on PostgreSQL didn’t look that fast, Sun (then Oracle) acquired MySQL, and the web demanded a different model! I even got swept up enough to write a small extension module for Django and Mongo.

Fast forward to today and MongoDB is backed by it’s own publicly traded company and you’ll find MongoDB experience in job postings and PostgreSQL and MySQL are… are probably bigger, in their own ways, than they were 10 years ago.

MongoDB’s performance gains turned out to have some tradeoffs, people realized that documents don’t free you from a schema, and PostgreSQL especially got even better. But guess what? There are even MORE databases now! People are still using PostgreSQL and MySQL and SQL Server, not to mention Redis and Elasticsearch, CockroachDB, Rethink, Riak, Cassandra…. document databases didn’t take over, they found a specialty slice of the market and the market simultaneously grew and segmented.

Yeah, if you have *actual* big data[2] or need to process real time data streams like time series, PostgreSQL may not be the best solution. But for probably 90% of uses cases it’s going to at least shine as your primary database or be the outright best solution.
 

Scaling in the “opposite” direction & closing thoughts

One of the situations in which I may not want to reach first for PostgreSQL would be dealing with local data where my query needs are simple and I never need to worry about multiple connections. That’s when SQLite would be a great idea.

Similarly I think Django’s job for serving simple content can be replaced one of the dozens of static site generators that have flowered over the past decade. There’s something truly wonderful about not having to worry about application servers or OS security when you absolutely don’t have to, and serving simple HTML content from a static source like AWS S3 is a blessing.

That said it’s naive to suggest that content management needs can be met using static site generators. Putting aside whether editing Markdown files and managing them in a Git repo is appropriate for a totally non-technical team, it doesn’t meet the content update requirements for most enterprise marketers.

A month or so ago someone asked me what I thought about Django as a framework, having worked with it for so long and I think by and large this captures my thoughts. In the future I see Django being augmented by additional technologies, rather than replaced.

I have a small side project that integrates with Slack and Harvest, our billing service. 8 years ago if I thought about turning it into a product I’d probably have thought about building a single Django project with a CMS for the marketing site, integrated into the entire rest of the application. Today I might extract the “public” facing components into a static site with Gatsby, use generic Python workers for scheduled Slack-based reports, a Django app for the core web-based customer experience, and Go for chat-integration.

It goes without saying that all of this could be done with Python, and with Django! But with different specialized tools at your disposal, they can complement Django projects beautifully.

Non-dogmatically yours,
Ben

[x] Disqus adopted Go for at least some backend processing (https://blog.disqus.com/trying-out-this-go-thing), Sentry started using Rust for processing source maps (https://blog.sentry.io/2016/10/19/fixing-python-performance-with-rust.html)
[1] Superbowl XXV https://en.wikipedia.org/wiki/Wide_Right_(Buffalo_Bills)
[2] Everyone thinks they have big data, but many folks just plain don’t. There’s no hard definition, but I favor the definition that it’s data that won’t all fit on one computer at once (many terabytes, if not petabytes).
 

More like this

• Django Is More Than Good Enough (This Old Pony #74)
• Hobble your Django app with complexity for fun and no profit (This Old Pony #72)