O3 Cyber Logo
See all posts

Kubernetes: Securing the Data Plane

Unlike the control plane, the data plane in Kubernetes (which includes your workloads and nodes) is user-managed in both on-premise and cloud environments. This means the security responsibility lies squarely with you. In this article, we share some essential practices to keep your data plane secure.

4 min•Feb 10, 2025
Andreas Døhl
Andreas DøhlLead Security Engineer
Andreas Døhl

Strict RBAC

Implement strict Role-Based Access Control (RBAC) policies following zero trust and least privilege principles. Apply these across namespaces, workloads, and the cluster to ensure minimal access.

Key considerations:

  • Be cautious with read roles. The Kubernetes verb list may seem less dangerous than get, but when paired with the -output switch, list can inadvertently expose sensitive details.
    • Example: Listing secrets or config maps containing sensitive information should not be allowed unless absolutely necessary.
  • Regularly audit RBAC policies to ensure they align with security best practices and operational needs. There’s tons of tools that can give you a good overview on your RBAC structure like this tool: https://github.com/alcideio/rbac-tool

Block Access to the Metadata Server (Cloud-Specific)

As mentioned in the "Securing the Control Plane" article, the metadata server is a common attack vector in cloud environments. It provides critical details about the infrastructure and often includes sensitive credentials, such as IAM tokens or API keys. Repeating this here is important because the data plane is where these risks often manifest.

  • Use network policies to block workloads that don’t require access to the metadata server—likely 99.9% of them don’t.
  • If left open, the metadata server can expose sensitive credentials, API keys, or information about the surrounding infrastructure, making it a critical risk to secure.

Make Use of Container Runtime and Access Control Policies

In Kubernetes, runtime and access control policies such as seccomp, SELinux, and AppArmor work together to enhance container security by limiting how containers interact with the underlying host system. These policies help prevent privilege escalation, unauthorized system calls, and other security risks.

Seccomp:

  • Filters and restricts dangerous or unnecessary system calls that containers can make to the kernel.
  • Protects against privilege escalation, code execution, and node compromise.
  • Ensure that a default seccomp profile is enabled for all workloads, and customize profiles for workloads with specific requirements.

Access Control Policies:

  • SELinux (for RHEL, Rocky, and other enterprise Linux distros):Uses mandatory access control to confine containers, preventing access to unauthorized system resources.
  • AppArmor (for Ubuntu):Provides resource access control by applying profiles that limit what files and capabilities containers can access.
  • On RHEL nodes, run SELinux in enforcing mode. On Ubuntu nodes, enable AppArmor and configure appropriate profiles.

Don’t use NodePort services

NodePort is a Kubernetes service type that exposes a pod to the outside world by opening a static port on every node in the cluster. For example, if you create a NodePort service on port 30080, traffic directed to <NodeIP>:30080 will be routed to your application.

The pros:

  • It bypasses the need for an ingress controller or external load balancer.
  • You can access the service directly via any node's IP address and the specified port.

The cons:

  • The NodePort is opened on all nodes in the cluster, even nodes that aren’t running the workload. This increases the cluster's attack surface, exposing applications directly without additional layers of protection.
  • It lacks advanced routing, rate limiting, and TLS termination features that ingress controllers provide.

For secure and scalable setups, it’s better to use an Ingress Controller or a LoadBalancer service type. Many modern solutions, like Cilium, have Ingress and API traffic capabilities built in, making them more robust and secure alternatives.

If you choose a LoadBalancer service, be aware of the allocateLoadBalancerNodePorts option. By default, Kubernetes will automatically allocate node ports for LoadBalancer services.

From the Kubernetes documentation:

"By default, spec.allocateLoadBalancerNodePorts is true, and type LoadBalancer services will continue to allocate node ports."

Ensure Image Integrity

Container images are the foundation of your workloads, so ensuring their security is critical:

  1. Use Image Scanners
    • Tools like Trivy can scan images for known vulnerabilities or exploits. Integrate these scanners into your CI/CD pipelines for automated checks.
  2. Leverage Private Image Registries
    • Store your images in a private registry with authentication enabled.
    • Use tags and hashes to ensure the correct image version is always deployed, reducing the risk of image tampering.
  3. Build Minimal Images
    • Start with a small base image (e.g., Alpine Linux) and include only the components your application needs to run.
    • This reduces the attack surface and limits the potential for vulnerabilities.

Wrapping Up

Securing the data plane is a critical aspect of Kubernetes security. By enforcing strict RBAC, blocking unnecessary access to the metadata server, applying default seccomp profiles, and ensuring image integrity, you can significantly reduce the risk of compromise.

Remember, the data plane is where your workloads live, and its security directly impacts your cluster's overall resilience.

To learn more about our Kubernetes Security offering: https://www.o3c.no/services/kubernetes-security-assessment