Data — specifically, monitoring and observability data that provides insights into system performance — is the fuel of IT operations engineers. Without that data, IT teams have little ability to detect performance risks, prevent applications and infrastructure from failing, or identify the root cause of an outage.
Unfortunately, collecting observability data has traditionally been a tough task. Although there are plenty of observability tools that IT engineers can leverage, most of them don't work in particularly efficient or user-friendly ways.
Fortunately, there's a better approach to observability for IT teams: the extended Berkeley Packet Filter, or eBPF. By providing a simpler and more efficient way to glean observability insights, eBPF can bring key benefits to IT operations.
Here's why ITOps teams should consider taking advantage of eBPF, and how they can do so.
What Is eBPF?
eBPF is a Linux kernel technology that makes it possible to run custom programs in what's known as kernel space. That means, in essence, that the programs run as part of the core operating system — as opposed to running in user space, where they are generic applications.
Using this approach, eBPF makes it possible to do two important things that are hard to do with conventional, user space observability apps:
- Collect valuable data about the status of any process or application running on a given server, because eBPF can "see" everything that the operating system can see — and the operating system can see everything.
- Collect data in a highly efficient way because programs that run inside the operating system consume minimal CPU and memory resources.
eBPF's Advantages for IT Operations
eBPF's ability to peer deep inside a server in an ultra-efficient way makes the tool valuable for a wide range of use cases — from security monitoring, to application debugging and tracing, to network performance management and beyond.
But one of the most obvious reasons to take advantage of eBPF is to streamline the monitoring and observability work of IT operations teams.
As noted above, ITOps engineers are typically tasked with understanding what's happening inside systems so that they can prevent and troubleshoot performance issues. But conventionally, the tools available to them for doing so were subject to important limitations:
- The tools consumed a lot of CPU and memory because they ran in user space.
- The tools had limited visibility because they were monitoring the system from user space. They lacked the all-encompassing vantage point of kernel space programs.
- In some cases, it was necessary to deploy multiple monitoring and observability tools — one for each application, process, or microservice you wanted to track — using the "sidecar pattern" because without access to kernel space, conventional tools couldn't track the entire system from a central location.
With eBPF, each of these limitations goes away. eBPF makes it possible to monitor every app or process on a system from a central vantage point, with minimal CPU and memory overhead.
In this respect, eBPF has the potential to bring enormous efficiency to both the workflows of ITOps engineers (who, with eBPF, no longer have to deploy multiple monitoring and observability tools or manage multiple observability data sources) and to infrastructure (which is no longer strained by having to host monitoring and observability software that sucks up CPU and memory, leaving fewer resources available to actual applications).
How to Use eBPF in IT Operations
Currently, there are two basic ways to go about using eBPF.
One is to write and deploy eBPF-based monitoring programs yourself. You can do this using tools like the BPF Compiler Collection, which contains command-line utilities that use eBPF to pull various types of monitoring and observability data from the Linux kernel.
To install the BPF Compiler Collection on a Linux server running Ubuntu or a similar distribution, simply use apt:
sudo apt-get install bpfcc-tools
Then, use the tools it provides to collect monitoring insights. For instance, if you want to monitor open() system calls, run:
sudo opensnoop-bpfcc
(Note that in most cases, eBPF requires root privileges by default, although it's possible to configure non-privileged eBPF mode if you want — and if you are willing to accept the security risks.)
Another way to get started using eBPF is to deploy monitoring and observability software that comes with eBPF built in. A growing number of observability tool vendors are now baking eBPF into their products such that they use eBPF in the background to collect observability data, without requiring users to set up eBPF themselves. This is a simpler way to take advantage of eBPF, although you'll be limited to whichever functionality the vendor decides to implement.
In the interest of avoiding appearances of vendor favoritism, we won't name any eBPF-compatible observability or monitoring tools here. But a Google search should reveal some names.
Conclusion
In short, eBPF is poised to make a tedious part of IT operations — monitoring and observability — much more efficient. The question facing IT teams is not whether they should take advantage of eBPF (the answer is a clear "yes" in most cases) but rather how to do it — and whether using eBPF will require them to adopt new tools or use in novel ways those they already have in place.
About the author
Christopher Tozzi is a technology analyst with subject matter expertise in cloud computing, application development, open source software, virtualization, containers and more. He also lectures at a major university in the Albany, New York, area. His book, “For Fun and Profit: A History of the Free and Open Source Software Revolution,” was published by MIT Press.
