Observability vs. Monitoring: Who Needs Which When?

Whereas monitoring was once the focus of teams tasked with managing software reliability and performance, the emphasis today is increasingly on observability--a concept that relates to, yet is distinct from, monitoring. Where does monitoring stop and observability start? Who needs observability today? Read on for a primer on what IT pros need to know about observability vs. monitoring.

What Is Observability?

Formally speaking, observability is the practice of analyzing the internal state of a system based on external outputs.

When applied to application performance management (APM), observability is the use of data that can be collected from the “surface” of an application--such as log files, metrics and transaction traces--to understand the system’s internal state.

What Problem Does Observability Solve?

In the context of software, the main benefit of observability is that it helps teams manage the performance reliability of highly complex systems, which would otherwise be difficult to monitor.

When you have a monolithic application that you deploy in a VM, it’s typically easy enough to track the health of the application using relatively basic data points, like memory and CPU usage. If you notice an anomaly within data from metrics like these, you can figure out what has changed--like which new code you added to the application, or which configuration changes you made to the host VM--to get to the root of the issue.

But when you have a microservices app, merely monitoring metrics like overall resource consumption is hardly enough to find and fix problems rapidly. You need to collect data from each microservice and correlate it to understand what is happening inside your complex system. That’s the only way to trace surface-level performance issues, like slow response rates, to the individual microservice or infrastructure component that is causing them.

How New Is Observability?

Observability itself is a relatively old concept. Its conceptual roots stretch back to the late 1950s,when observability emerged as a concept in the field of control theory.

It was only in the past decade, however, that observability has become a trendy concept within the realm of APM. That’s thanks to efforts by engineers at companies like Twitter to promote observability as a means of understanding what is happening inside highly complex, distributed systems--such as those based on microservices architectures--which have also become widespread in the past decade.

And it was not until about 2017 that observability really started coming into vogue, with APM vendors rushing to rebrand their tools as “observability solutions” rather than mere monitoring or APM platforms.

Observability vs. Monitoring: What’s the Difference?

The extent to which the surge of interest in observability reflects something that is actually distinct from monitoring is debatable.

If you google “observability vs. monitoring,” you’ll find a few leading perspectives on the differences between these two concepts and practices. They basically boil down to the following:

• “Monitoring is the actual task of collecting … data,” as Instana puts it, while observability focuses on using the data.
• Monitoring explains what’s happening, and observability explains why. That’s the crux of Devo's perspective.
• Monitoring lets you track problems you already know could exist--in other words, “known-unknows”--whereas observability helps you think more broadly and creatively, revealing “unknown-unknowns.” This is how Charity Majors, one of the leading figures in the observability discussion, describes the differences between observability and monitoring.

This is a somewhat simplistic overview of different takes on the observability vs. monitoring question. If you read through the sources cited, you’ll see that they offer more nuance than the summaries above.

Still, what’s clear is that different people offer surprisingly different definitions of observability and monitoring. That makes the whole discussion surrounding observability suspect to a certain degree. If we can’t even define the differences between observability and monitoring clearly and consistently, is observability really that different from monitoring?

Who Needs Observability?

That brings us to the ultimate question, which is whether your team or business actually needs observability.

Obviously, the answer depends on how you define observability and its differences from monitoring--which, as we just noted, is a messy topic.

But, in general, perhaps the best way to think about whether you need observability is to evaluate your current monitoring and APM strategies and determine whether they are meeting your needs.

• If you are struggling to understand the state of your distributed systems because your existing monitoring data is too siloed, you may need observability.
• If you find that you have lots of monitoring data, but struggle to use it rapidly to solve actual problems, you may need observability.
• If your team is too focused on “known-unknowns” to think creatively about performance or reliability issues that haven’t happened before but may arise in the future, you may need observability.

In other words, if your current monitoring or APM strategy isn’t cutting it in some way, then plugging into the discussion and tools surrounding observability may be worth your while.

On the other hand, if you’re meeting your SLAs and delighting your customers with whichever APM strategy you currently have in place, more power to you. You probably shouldn’t waste time playing with new tools or new buzzwords to solve problems that don’t exist for you.