The Surveillance Software That’s Secretly Running on Your Kubernetes Cluster

Kubernetes teams spend countless hours worrying about what goes into their clusters. Far fewer consider what quietly leaves. Today, many Kubernetes workloads are constantly producing telemetry, logs, metrics, traces, configuration, API events, and runtime data. While organizations use this to power their observability stacks for debugging applications and maintaining services, it begs the question: what’s being transmitted to third-party services, and who has access?

As companies embrace cloud-native software, monitoring agents go largely unnoticed. They are installed to enhance observability, security, performance, or compliance, and typically go on for years unreviewed. The majority of these agents are legitimate, but some may end up sending far more operational data to external services than organizations know about.

Observability vs oversharing

A typical Kubernetes cluster often runs more than applications. Alongside logging agents, you have metrics collectors, service meshes, runtime protections, cost monitoring, CI/CD integration tools, and more. While each may have a legitimate reason to be there, a lot of them require privileged access to the cluster.

Over time, these agents tend to accumulate. One team installs a monitoring tool, another installs a security platform, a third installs a cost optimization solution, and few stop to check exactly what data is being collected or where that data goes.

Observability needs data. To diagnose issues across a distributed system, companies need to know their applications’ logs, metrics, and traces. The question is how much data they really need.

Data collected by observability tools is typically broad because more data usually results in better analytics and troubleshooting. This information may contain Kubernetes objects, namespaces, deployments, clusters, infrastructure, and more. It can also include application log samples.

Why Kubernetes makes this easy

In highly regulated industries, data about infrastructure and deployment patterns could be considered confidential or sensitive. Even telemetry data that doesn’t contain business secrets can expose information about infrastructure architecture, internal applications, and customer data.

Kubernetes was designed to make infrastructure observable. The API exposes rich information about application workloads, events, clusters, pods, containers, namespaces, and labels. That is one reason why the ecosystem has become so powerful.

It is also what makes agents so much more capable. If they have the necessary permissions, agents can continuously scan for cluster changes, track the status of running services, detect new applications, monitor deployments as they scale, and gather metadata about clusters. This becomes problematic.

Detecting unauthorized data collection

The first challenge is knowing what is actually running. In many cases, organizations lack a complete inventory of monitoring and telemetry systems across all Kubernetes clusters. Shadow tooling shows up during proofs-of-concept projects, acquisition efforts, or troubleshooting purposes, eventually becoming a permanent fixture.

Visibility starts with inventory. The platform team should periodically check for all DaemonSets, sidecar operators, and privileged workloads deployed within a cluster. Outbound connections should also be inventoried. Knowing which services communicate with external endpoints helps identify software that continuously exports telemetry.

RBAC should also be reviewed for all monitoring systems. Systems with a full view of a cluster’s resources should have a reasonable explanation for such access. Systems that have been granted excessive permissions or permissions that include wildcard resources or API access increase security and privacy concerns.

Organizations should also periodically audit the telemetry itself. Understanding what logs, metrics, traces, and data leave the environment can reveal unnecessary collections. This can help organizations reduce the amount collected with very little change to operational visibility.

Governance is as important as security

This discussion is starting to change from just a cybersecurity discussion to a governance discussion. Platform engineering teams should have clear policies defining what observability systems should be installed, what information they should collect, where the telemetry should be processed, and how long it should be retained.

This is even more important as organizations begin to adopt AI-powered observability solutions that process massive volumes of observability data to make recommendations and solve issues.

Without governance, it is easy to begin to over-collect. Observability agent software should be treated like any privileged workload to maintain trust and reduce unnecessary exposure.

Finding the right balance

None of this is an argument against deploying observability agents to a Kubernetes cluster. We simply cannot operate in cloud-native environments without comprehensive monitoring. The objective here is proportional visibility. Collect data that is required to ensure reliability, security, and performance, but don’t collect data simply because you can or because the platform allows you to. Every observability stream should have a defined use case, retention policy, and data ownership.

Periodic review of what observability systems have been installed, what RBAC permissions have been assigned, what outbound connections have been established, and what data has been collected will ensure that observability is operating as an asset and not as a source of security issues. One of the most significant sources of surveillance potential within a Kubernetes cluster is not malware. It’s trusted tools that have been granted broad access. When more systems and tooling are added to the Kubernetes cluster and more agents are deployed in the environments where those tools are running, understanding what data they collect and where it is sent is just as important as knowing what tools are deployed to the cluster.

The future of Kubernetes observability will not be determined by how much telemetry is collected. It will be determined by collecting the right telemetry with the right access, for the right reasons.