How Ambient Mesh Is Changing Kubernetes Network Architecture

Picture this. Your platform team just spent six months rolling out a service mesh. Latency went up, memory usage spiked, and every pod now carries a sidecar proxy that nobody fully understands. Sound familiar? This is the exact pain point that pushed the Kubernetes networking world toward something new called ambient mesh.

In this article, you will learn what ambient mesh actually is, how it ditches the sidecar model without giving up the benefits of a service mesh, where it fits into your existing Kubernetes setup, and what tradeoffs you need to weigh around security and performance before adopting it.

What problem is ambient mesh actually solving

For many years now, the common method to operate a service mesh such as Istio has been through the injection of a sidecar proxy – generally, Envoy – into every pod. This sidecar proxy intercepts traffic and operates from there, enforces mutual TLS, enforces policy, and collects telemetry data. This method has worked flawlessly, but it has definitely been costly; every pod has gotten heavier; every deployment has become more complicated; and resource usage has scaled linearly with the number of pods, as opposed to the traffic load.

As teams began operating large clusters containing thousands of pods, they began feeling the pain of the aforementioned sidecars more and more. Sidecars consume CPU/Memory, even when there is not much traffic between them. Upgrading the mesh requires restarting all of the pods for the purpose of re-injecting with the new proxy version. Debugging has become more difficult since there are now two containers to troubleshoot for each workload.

Ambient mesh has been invented for the express purpose of eliminating that sidecar cost, while still providing the same mTLS, traffic management, and observability services that service meshes originally provided their users.

How ambient mesh actually works

Ambience mesh separates the data plane into two separate layers, rather than wrapping everything together in a single sidecar at each pod.

The first layer is called the “secure overlay” layer and is controlled by an element commonly referred to as ZTunnel (also called “Z Tunnel”). This is a lightweight node-level proxy that operates at one node instead of being operated on every pod. This element is responsible for handling all of the core components of service-to-service communications, including: simple Layer 4 routing capabilities, mutual TLS/SSL encrypted traffic, and mutual identity based authenticated traffic between workloads. Since ZTunnel is located at the node level, there is no requirement to inject it into every pod, and it does not restart your workload when an upgrade is performed on the mesh.

The second layer is referred to as layer 7 processing, and is handled by waypoint proxies. Waypoint proxies are optional and only deployed for namespace or service that requires more advanced capabilities such as; HTTP routing, retry logic, circuit breaking capabilities and fine-grained authorization policies. Therefore if a particular service only wants simple mTLS/SSL encrypted traffic and has no need for Layer 7 capabilities, it will not ever communicate with the waypoint proxy.

This division of processing capabilities at the node and service level is where the true innovation occurs. With the node-level proxy handling the basic security and authentication requirements for every workload, you have the ability to get both baseline security and identity for every workload without incurring costs associated with layer 7 processing for workloads that do not require them.

The operational benefits that actually matter

The biggest benefit is that resource efficiency is improved. Since the ztunnel only runs once per node instead of once per pod, the overhead from memory and CPU is scaled to the amount of nodes you have rather than to the amount of pods. This can result in a significant decrease in resource usage for clusters running dozens of pods per node.

Upgrading your mesh becomes easier as well; because you will no longer need to bake sidecars into each pod, updating the mesh’s data plane doesn’t require you to restart any of your application workloads. This eliminates one disruption when you are operating a mesh at scale.

Existing workloads can be onboarded with less disruption than before because the addition of namespaces to the mesh does not require changes to the deployment manifests or concern for the success of sidecar injection webhooks failing silently. Therefore, legacy applications that were considered too risky to mesh before now have a lower barrier to entry into mesh networks.

Platform teams will have more clearly delineated areas of responsibility. Security and identity will become a foundational layer of a mesh that is governed and managed centrally, whereas Layer 7 traffic policy will be able to be optionally implemented by application teams when they feel it would provide value. The flexibility in adopting layers of mesh technology has the potential to decrease much of the all-or-nothing type of barrier many teams experience when considering adopting a mesh network in general.

Where ambient mesh fits into modern Kubernetes strategy

Ambient mesh is not a replacement for every sidecar mesh use case overnight. It fits best in environments where you have a large number of services that mostly need basic encryption and identity, with only a subset requiring advanced traffic management.

If you’re running a platform with hundreds of microservices, many of which are simple internal APIs, ambient mesh lets you secure all of them by default without forcing every team to understand or configure waypoint proxies. Then, for the services that genuinely need canary deployments, request level retries, or detailed authorization rules, you layer in a waypoint proxy selectively.

It also plays well with multi tenant clusters. Since the node proxy handles identity and mTLS uniformly, platform teams get consistent security posture across tenants without depending on every application team to configure their sidecar correctly.

That said, if your workloads are heavily dependent on Layer 7 features across the board, the operational savings from skipping sidecars everywhere shrink, since you’ll end up deploying waypoint proxies broadly anyway.

Security considerations worth weighing

The Ambient Mesh provides strong mTLS-based identity to all workloads through the ztunnel layer, so overall security is equivalent to using a regular sidecar mesh. Every pod has an identity that is cryptographic and uses encryption for all traffic by default.

However, implementing security policies is more difficult due to the reliance on only the waypoint proxy to enforce Layer 7 authorization (e.g. restricting access based on HTTP path or header) because if a service does not have a waypoint, it will be limited to Layer 4 controls.

Teams should be intentional about which services require Layer 7 inspection, and only deploy waypoints where required, so that there is no gap between perceived policy enforcement and the actual implementation of that policy.

Because the node-level proxy will be shared between pods on that node, a ztunnel compromise will have a blast radius that is greater than what was previously experienced with per pod sidecars. Therefore, improving the security of node-level components and ensuring that multi-tenancy of nodes is strictly controlled will be beneficial when using this model.

Performance considerations worth weighing

Removing the sidecar hop for Layer 4 traffic generally reduces latency overhead for services that don’t need Layer 7 features, since traffic goes through one node level hop instead of two sidecar hops per connection.

For services that do require waypoint proxies, the latency profile becomes closer to traditional sidecar mesh performance, since traffic now passes through both the ztunnel and the waypoint. The performance win is really concentrated in the services that can stay at Layer 4, so the actual gains you see will depend heavily on your service mix.

Resource consumption, as covered earlier, scales better with ambient mesh because node level proxies don’t multiply with pod count. This generally translates into lower overall cluster resource usage, especially in high pod density environments.

Is it time to make the switch

Ambient mesh provides an authentic architectural evolution in the way Kubernetes networking is able to enforce security and manage traffic as opposed to being simply another “lighter weight” implementation of the same concept. For example, by decoupling identity and encryption from Layer 7 policy, which gives platform teams the ability to enforce all security policies and provide advanced options that are terms of engagement.  

Whether or not your team has been hesitant about trying a service mesh due to sidecar burden, or whether or not you’re already utilizing sidecars and are feeling overwhelmed by their operational overhead, we suggest running an evaluation of ambient mesh in a non-sensitive namespace to test true traffic patterns, document differences in resource utilization, and determine where waypoint proxies really provide benefits for your application.  The architecture is advanced enough now to warrant serious evaluation and the potential operational benefits for most clusters are hard to dismiss.