You know Kubernetes can orchestrate workloads that are distributed across clusters of servers in a single cloud or data center.
But did you know Kubernetes can also manage clusters spread across multiple clouds or data centers? That’s what you get when you run multi-cluster Kubernetes, which is fast becoming the latest, greatest way to get more value out of Kubernetes, the open-source container orchestrator.
Here’s a primer on how multi-cluster Kubernetes works, its benefits and drawbacks, and how you can build a multi-cluster environment.
What Is Multi-cluster Kubernetes?
Multi-cluster Kubernetes is a Kubernetes environment that includes two or more clusters managed through a single Kubernetes control plane.
This is different from running multiple namespaces within the same cluster. Multiple namespaces let you isolate workloads within the same cluster, but they don’t give you the flexibility to deploy workloads across multiple sites, as you can with multi-cluster Kubernetes.
There are several ways to implement a multi-cluster setup:
- API centralization: You can use a framework like KubeFed to manage multiple clusters through a single API server that can control multiple clusters.
- Kubelet-based: Projects such as Virtual Kubelet and Tensile-Kube make it possible to manage multiple clusters through a single control plane even if each cluster uses its own APIs.
- GitOps: It’s possible to manage multiple clusters centrally using GitOps.
Thus, multi-cluster Kubernetes refers to an architectural style rather than a specific type of solution or platform. It’s like multicloud or hybrid cloud in that respect: There are many ways to achieve it.
The History of Multi-cluster Kubernetes
Until recently, multi-cluster Kubernetes had been much-discussed but little-implemented.
Early efforts to support multiple clusters through a single control plane experienced a setback when the original implementation of Kubernetes federation was scrapped in late 2018. It took some time to build the new version, KubeFed, which even today remains in beta.
But given the increasing stability of tools like KubeFed, as well as the emergence of other solutions for multi-cluster management, there has been an uptick of interest in multi-cluster Kubernetes architectures over the past year or so, especially after the Cloud Native Computing Foundation published a blog post promoting the concept last April.
Why Use Multiple Clusters?
The growing interest in multi-cluster Kubernetes reflects a realization that, for many workloads, two (or more) clusters are better than one, for several reasons:
- Resiliency: When you run multiple clusters in different physical locations, you increase your ability to keep operating even if one of those locations is taken offline. A data center fire that disrupts one of your clusters won’t bring down your entire environment.
- Performance: Multiple clusters make it easier to place workloads close to end users because you can locate each cluster in a data center or cloud region close to a different concentration of your end users. The result is lower network latency and a better user experience.
- Security: Multi-cluster Kubernetes allows you to isolate workloads in entirely separate clusters, providing more security insulation than you can achieve via namespace-based isolation.
- Multi-cloud management: If you use multiple clouds, multi-cluster Kubernetes makes it possible to run clusters in each cloud without having to manage them separately. In this respect, multi-cluster Kubernetes is a great way to simplify the management of multiple clouds.
- Centralized management: Combining multiple clusters with a single control plane makes it easy to manage highly distributed infrastructures. That could be particularly useful for edge computing use cases that involve small sets of servers or internet of things (IoT) devices spread across a wide geographic area.
The Challenges of Multi-cluster Kubernetes
While multiple clusters provide a number of benefits, they also make Kubernetes – which is not exactly easy to manage even when you have just one cluster to deal with – even more complex.
Probably the biggest challenge is networking. If you have multiple clusters located in different data centers or clouds, it’s likely that the networking configurations and tooling for each site will be different. Engineers need to figure out how to reconcile routes, IP addresses and so on between each cluster.
Tools such as Liqo-Route can help with this, as can service meshes like Istio and Linkerd that are designed for multi-cluster support. But even with the help of these tools, network design and monitoring require a lot of effort when you have two or more clusters connected to a single control plane.
Security is also a challenge. You’ll need to make sure that policies such as Kubernetes security contexts let you enforce access controls properly across multiple clusters. That can be tricky, especially if you run into situations where resource names are the same in each cluster.
As long as you plan properly, however, it’s likely to be easier in the long run to manage two or more clusters via a single control plane than it would be to manage each cluster separately.
That’s why it’s likely that we’ll be hearing more and more about multi-cluster Kubernetes in 2022. Just as single cloud now feels old-fashioned, running just one Kubernetes cluster may become outdated as organizations look for ways to leverage Kubernetes more efficiently across complex infrastructures.