Sometimes things don’t work exactly as planned.

That’s just something you have to take as given, on the road and when you’re operating a production web application.

Instead the question is, how do you know what’s going on? 

It’s fair to say the driver of that truck has a pretty clear understanding now of what happened, when, and hopefully why. But when a Django application is running in a data center somewhere and customers are interacting with it, you don’t have that same level of obvious feedback.
 

Logging, not just for lumberjacks

No need to bury the lede too far, the answer is that you need to use logging.

For those unfamiliar with application logging, it is very much like it sounds. It’s a way of using messages sent from and outside of the application in a stream (often to a file) with timestamped messages or structured data to log what happened when an application was running.

Logging in it’s various forms, allows us to learn different things about an application that aren’t always obvious:

• ​Errors, which includes exceptions
• Performance
• Status of out of band task
• Code usage More fundamentally, the purpose of logging is to provide information, information that helps us answer questions, make decisions, and act.
• What went wrong? Why?
• How long does a response take?
• How long do various components of the response take?
• Are these tasks completing?
• How much is this module actually called in production?
• What are people actually doing on the site?

Deciding where to start adding logging

Before you start adding logging statements, the place to start is your questions.

• What do you not know about your application?
• What isn’t or seems like it isn’t working correctly in production?
• What untested assumptions do you have about what it - or your customers - are doing? As it happens, there are a few common places to start based on these questions.
• try/except blocks: if you’re not sure exactly why a certain exception would occur _and _it’s not a totally normal part of the programmatic flow (e.g. using an exception to signal type coercion), then that’s a good candidate for logging
• Anything run out of the request cycle: these could be async tasks or cron scheduled management commands
• Business critical transactions: things like subscription sign ups and cancellations (there’s another way to handle these though that we’ll cover shortly)
• Third party API or webhook interactions: this is critical because there’s little other recourse in debugging problems with a service you don’t control  

  How much is too much?

Log data should be informative. To that end, you want signal, not noise. Like useless or bad tests, excessive log entries make it harder to find what you’re looking for.

That being said, when your’e starting out err on the side of more rather than less. Pare down logging later, rather than holding back on what should be logged.

Logging vs. monitoring

There’s some overlap between both the purpose and coverage of logging on the one hand and monitoring on the other. They are different things however, and one does not preclude the other. In fact, a monitoring system may be keyed off your logging data.

The main purpose of monitoring is to proactively alert you to issues with your application. This is different than logging in that logging is _primarily _a tool for assessing after the fact. This true even in the presence of real time logging. But if nothing logging, by itself, requires you to come to it, and monitoring is designed to proactively reach out.

With regard to performance, most monitoring solutions provide something like response time analysis, but don’t always provide the kind of deep reach into your code to find out how long, say, one specific function in a module is performing. Most. Some do, and when available take advantage of these tools! They can be quite expensive, and if you really only care about one area of your code, the right logging will give you what you need.

To repeat: you want logging even when you have a monitoring solution in place.

STDOUT yours,
Ben

More like this

• How logging, monitoring, and analytics work together (This Old Pony #20)
• This Old Pony: Monoliths or Microservices?