There is profound change brewing in the world of cloud computing, and it can be summed up as a four-letter word: eBPF.
Indeed, eBPF — the open source technology that makes it possible to run hyper-efficient programs in the Linux kernel — has enormous potential for upending the way customers experience public cloud services.
Unfortunately, though, public cloud vendors have been slow to react to the eBPF revolution. Although it's not impossible to use eBPF and eBPF-based tools in conjunction with public cloud workloads, it's harder than it should be
That needs to change to ensure that eBPF doesn't pass the public clouds by and deprive their customers of the substantial observability, security, and other benefits they stand to glean from eBPF.
To prove the point, let's walk through what eBPF means, why it's so important for public cloud platforms, and what the public clouds should be doing starting now to take full advantage of eBPF.
What Is eBPF?
The Extended Berkeley Packet Filter, or eBPF, is a technology that enables the execution of sandboxed programs within the Linux kernel.
Put another way, eBPF makes it possible to run low-level software directly within the Linux kernel — as opposed to within "userland," the space where Linux programs typically run.
That's a big deal because programs that run in kernel space enjoy a few very important advantages over conventional programs:
- When properly designed, they can "see" everything the kernel can see, which means they can collect system performance, security, and other data that would be impossible or very difficult to collect from userland.
- They consume very few resources when running, making them much more efficient than conventional software.
- They can be loaded dynamically, which means users can write and run eBPF programs of their choosing whenever they want. They don't have to compile the code into the kernel or rely on kernel modules.
- Because eBPF programs are sandboxed from each other and from other kernel resources, they are very secure.
These eBPF benefits add up to a technology that makes it possible to collect much more information from Linux servers, in a much more secure and flexible way, while consuming many fewer resources.
eBPF and Public Cloud Computing
eBPF works on any server (or, for that matter, PC) that runs a modern Linux kernel. It doesn't matter whether the server runs on-premises or in a public, private, or hybrid cloud.
However, eBPF offers particularly exciting potential when applied to public cloud-based servers. The main reason why is that eBPF could solve what has long been one of the biggest drawbacks of public cloud infrastructure: a lack of visibility into the cloud servers on which cloud-based workloads run.
Traditionally, businesses that want to use the cloud have had strictly limited visibility into the state of the cloud infrastructure they use. They are only able to collect whichever log and metrics data the cloud vendors choose to expose.
eBPF, however, could change this by allowing cloud customers to write eBPF programs to collect whichever data they want from host servers. Because the programs are completely customizable, customers would not be restricted to generic observability or security metrics. And because eBPF programs are highly secure, the risk that they would create security gaps is relatively low.
Plus, for cloud users who don't want to write their own eBPF programs, robust eBPF support would make it easier to take advantage of third-party monitoring and observability platforms — such as Datadog and New Relic, to name a couple — that are now leveraging eBPF to help collect observability data.
The State of eBPF in the Public Cloud
It's not impossible to take advantage of eBPF on the public cloud today. Some public cloud services — especially managed Kubernetes services like EKS — support third-party plugins, like Calico and Cilium, that use eBPF to collect observability data. It's also possible to enable eBPF on cloud-based VM instances by using custom kernels.
However, public cloud providers have yet to treat eBPF as a first-class citizen. AWS has so far resisted requests to provide AMI images that support eBPF by default, for example. Microsoft has announced a project to bring eBPF-like technology to Windows but hasn't said much about how it might use eBPF in Azure. Google has invested in eBPF integrations for GKE, its managed Kubernetes service, but there are no signs that it is thinking about how to give customers easy access to eBPF-based tools or integrations across other types of cloud services.
The lack of native support for eBPF on most public cloud services poses a challenge for customers because eBPF is not something that can be turned on once workloads are running. It needs to be enabled in the kernels that host workloads when servers boot. By not offering eBPF by default, cloud vendors force customers to build and provision custom kernels if they want to take advantage of the technology.
What all of the above means is that, if you want to take advantage of eBPF to help monitor, observe, or secure workloads running in a public cloud today, you'll more likely than not have to do a lot of custom setup. Or maybe you'll decide to keep your workloads on-premises, where it's easier to enable eBPF because you have full control over the infrastructure and what runs on it. The cloud providers simply aren't doing much to make it easy for their customers to use eBPF.
Here's hoping that changes — and soon — because it's increasingly clear that eBPF is poised to transform the way IT organizations manage workloads. The longer it takes cloud vendors to get on board with the eBPF revolution, the greater the risk that they'll be left behind by customers who are disappointed by cloud providers' half-hearted embrace of eBPF.
About the author
Christopher Tozzi is a technology analyst with subject matter expertise in cloud computing, application development, open source software, virtualization, containers and more. He also lectures at a major university in the Albany, New York, area. His book, “For Fun and Profit: A History of the Free and Open Source Software Revolution,” was published by MIT Press.
