This is a slightly longer email than normal and well worth the read.

[Logging] is a way of using messages sent from and outside of the application in a stream with timestamped messages or structured data to log what happened when an application was running.

We briefly touched on the difference between monitoring and logging under the presumption that they have strong overlap. And they do. Both involve capturing information about your application to be assessed in real time or historically.

There are a few other related pieces though and its important to understand how they all fit together.

1. Monitoring, primarily concerned with the _state _of your application
2. Error tracking, a specialized form of logging capture that 
3. Analytics, primarily concerned with user behavior and measuring general aspects of the UI
4. Metrics, primarily concerned with customer behavior and the performance of business processes
5. Logging, primarily concerned with activity _inside _the application

These are some broad brushstrokes and there’s a lot of overlap, but some distinction is necessary to understand how to use these.

For each we’ll examine the use case, questions it answers, stakeholders, and how its relevant to maintaining a Django application.

Monitoring

Monitoring is mainly concerned with the _state _of your application. It’s like the instrument panel on a car. It should tell you if the engine is running, how fast you’re going, how fast the engine is turning, how hot the engine is, how much gas you have, and whether there are any engine problems.

That’s state at a glance plus alerts.

Every application of non-trivial consequence should have some kind of monitoring in place. This can range from pinging the server for a 200 response to integrating a complete performance monitoring system[0].

Key is that monitoring is predicated on a system separate from your application.

This is what you want for questions like:

• Is the application running?
• Are all components of the stack up?
• Is, or what part of, the application stack is slow? It’s instructive to add monitoring to an existing Django app because it can reveal problems previously hidden and also show what’s _really _causing problems you think you have. E.g. APM (application performance management) integration can reveal that your database isn’t the problem, but instead how your queries are structured, potentially saving significant wasted time chasing the wrong problem.

The primary stakeholders are developers and systems administrators (or anyone wearing those hats).
 

Error tracking

If you’re still tracking errors using Django’s admin mail, it’s high time you took a look at options for error tracking.

Error tracking uses Python’s logging system to send your detailed error messages _and exception stacktraces _to a separate system where they can be examined by your team, triaged, logged in engineering management systems, and all without flooding inboxes.

This is what you want for questions like:

• Has something gone wrong in the application?
• What exactly is the problem? (e.g. where in the code)
• What is the context for this error? The primary stakeholders are developers and customer support.

There are excellent commercial offerings as well as open source solutions[1].
 

Site analytics

It doesn’t have to be Google Analytics, but we all know it probably is.

At a base level, website analytics tell you how much your site is being accessed. How many visits, how many new visits, which pages, for how long, what are your site’s main entry points, main exit points…

This is what you want for questions like:

• Which pages or routes are accessed?
• What devices are customers using?
• Which browsers are customers using? The primary stakeholders are product owners, marketing teams, and yes, even developers. Some tools can provide significant insight into page performance which monitoring tools integrated into the backend may not be able to provide.

If you’re not measuring some kind of use of your site, you should be. A significant benefit for maintaining a Django site is knowing what parts of your site are being used, whether your site is a product or a publishing channel (e.g. CMS).
 

[Product] Metrics

Metrics is something of a catchall but most “metrics” oriented tools are focused on the other side of the coin from monitoring, which is the user or customer side. They help measure how your site is performing as a product, before and after customer acquisition[2].

We’ve specifically chosen the word “customer” in the intro not because metrics can’t refer to anything else - like application metrics - but to call out a significant and _different _use case. Whereas the use of general purpose analytics tools is more page oriented, the use of metrics tools tends to be more _person _oriented.

This is what you want for questions like:

• What are our conversion rates?
• Which kind of customers are we retaining?
• Which campaigns had the highest net return?

The primary stakeholders for metrics include “growth” teams and product owners(and yes, developers).

The value of adding metrics like this to a Django site is premised on the idea that decisions made in a vacuum are usually bad. What would you do differently if you knew that one seemingly minor issue was influencing customers not to renew? Or that a costly feature to repair actually has little drag on the business?

Some site analytics tools can support this kind of use case, but doing so requires significant user savviness. More focused tools are a better option for sites with even somewhat sophisticated needs. Moreover, many metrics tools can be integrated into your local customer model where other tools like Google Analytics cannot[3].

Logging

It’s entirely possible that you could answer most of the questions enumerated above using only application (and backing service) logs, provided you’re producing sufficiently detailed logs in your Django app.

And doing so would enormously inconvenient and inefficient.

The tools here provide more than simple roll ups of the data, and between the sophistication of tools and differences in stakeholder needs, it’s better to have some combination of different tools for these varied information sources.

The lowest hanging fruit with good returns, if your Django site is missing them is analytics on the front-end and error tracking on the backend. This gives you a _ton _of information with very little effort and minimal code changes required. At least with Sentry, you can capture _error _logs as well as exceptions, which should give you access to the highest priority data.

From there consider adding monitoring, even basic monitoring, and then ensure you have a system in place to capture logs (to mention your loggers configured).

Customer and product metrics are significant value additions but as a rule should follow the rest. How you weigh the complexity of each will differ from scenario to scenario. For a complex, high traffic app serving a single customer, it could be worthwhile integrating performance monitoring earlier, whereas for a SaaS pivoting its business model, basic monitoring should suffice and attention instead given to serious customer metric strategy.

We’ll likely take a detour to a different topic next week, but we’ll be back to discuss some hands on logging intervention.

20171024 - ThisOldPony - INFO - Signing off,
Ben

[0] Commercial examples include NewRelic, Datadog, open source solutions include Nagios, Munin
[1] Sentry is a great commercial product as well as open source option.
[2] A few options here include Kissmetrics, Mixpanel, Keen.io - Google Analytics also works and for simple needs you can roll your own solution use your own DB or, say, ElasticSearch
[3] You need to be very careful about what information you provide to Google Analytics, their ToS specifies that they do not want PII of any kind through this service

More like this

• The point of no return and the advantages of web apps (This Old Pony #21)
• This Old Pony: What's going on!? Or, a little bit about logging