In the following weeks, I’ll provide insights into Kubernetes security and architecture to help you secure your container infrastructure and understand what’s happening under the hood. Whether you’re new to Kubernetes or looking to deepen your expertise, there’s something here for everyone. Let’s dive in!
Instead of overloading you in the first post, let’s break down the foundational components of a Kubernetes cluster. Whether you’re running Kubernetes on-premises, leveraging Kubernetes-as-a-Service in the cloud, or using a hybrid approach, the core components remain the same. By understanding these, you’ll build a solid foundation to grasp the key concepts, no matter which platform you’re working with.
1. Nodes
- A Kubernetes node is a single machine (physical or virtual) that runs workloads in the cluster.
- Nodes are categorized into:
- Control Plane Nodes: Responsible for managing the cluster and making decisions about scheduling, state, and configuration. This is where the most sensitive components run, making them the ultimate target for attackers.
- Worker Nodes: Where your actual applications and workloads run. While less critical than the control plane, misconfigurations here can lead to non-intended data exposure, lateral movement and potentially compromising other components.
2. Pods
- A Pod is the smallest deployable object in Kubernetes.
- Each pod usually encapsulates one or more containers and provides them with a shared network and storage context.
- At the end of the day, most workloads in Kubernetes are essentially one or more pods, whether they’re managed by DaemonSets, Deployments, StatefulSets, or (Cron)Jobs.
3. Namespace
- A Kubernetes namespace is a logical partitioning within the cluster, designed to help segment applications and environments. Think of it as a "resource group."
- In environments with multiple teams or users, namespaces allow segmentation by giving each user or team their own space to deploy resources.
- Using namespaces, combined with RBAC (Role-Based Access Control), you can restrict access, ensuring users can only view and control resources in their own namespace.
4. The API Server
- The API server is the first point of entry to the Kubernetes cluster.
- It processes all requests (e.g., kubectl commands) and ensures they comply with the cluster’s policies.
- A compromise of the API server is essentially “game over” for the cluster. This is because it is where the RBAC and Pod Admission controls are located. More on this later.
5. etcd
- At the heart of Kubernetes is etcd, a distributed, highly available key-value store.
- etcd stores the desired state of all cluster objects, such as pods, services, and deployments.
- Importantly, only the API server communicates with etcd to ensure consistency and control. A compromise of etcd is also “game over”, as it serves as the authoritative source for the cluster’s state at all times.
6. The Scheduler
- The Scheduler decides where to run a pod by analyzing resource availability and constraints across nodes.
- Once a node is chosen, the API server updates the pod’s metadata in etcd to reflect this decision.
- A compromise of the scheduler isn’t nessecary critical, but it can give the attacker control over where workloads are placed in the cluster.
7. Kubelet
- The kubelet runs on each node and ensures that the containers specified in the pod definition are created and running.
- It communicates with the Container Runtime Interface (CRI) and Container Network Interface (CNI) to pull images, create containers, tell the CNI to create network connectivity and manage resources.
- Since the kubelet normally runs as a service on the underlying node, it’s a bit harder to get access to. But still, it has an API where it’s possible to disable authentication, so misconfigurations can happen.
8. The Container Network Interface (CNI)
- The CNI provides network connectivity within your Kubernetes cluster.
- It also enforces Network Policies to control traffic between pods and external systems.
- Key Risk: Overly permissive Network Policies can allow unauthorized communication and expose sensitive resources.
9. Kube Proxy
- The kube-proxy is the network glue of Kubernetes, managing how traffic is routed between services and pods. It’s also a developer’s best friend when it comes to debugging and testing applications.
- For example, if you want to test whether your web application is running correctly, you can use kubectl port-forward to expose the pod on your local machine, making it accessible at http://localhost:8080. This is incredibly useful for verifying applications without needing to expose them to external networks.
- This also comes with security risks. The traffic normally runs in plain HTTP, so an attacker can sniff the traffic if he is able to access the node serving the Pod being port forwarded. You can also expose sensitive components like the API server (or even the etcd server), so make sure to clean up any active port forwards.
To wrap this up
- API Server – The main entry point for all cluster operations. Compromising it is essentially game over since it controls authentication, authorization (RBAC), and admission policies.
- etcd – The brain and storage of the cluster, holding the desired state. If etcd is compromised, the attacker effectively owns the cluster state.
- Control Plane Nodes – These run the API server, etcd, the scheduler, and controllers, making them a high-value target for attackers.
- Worker Nodes & Pods – The actual workloads run here. If compromised, an attacker could laterally move within the cluster, potentially reaching control plane components if not properly secured.
- Kubelet – Manages pods on each node. If exposed or misconfigured, it can grant unauthorized access to container workloads or their underlying node.
- CNI (Container Network Interface) – Critical for pod-to-pod networking and enforcing network policies. Misconfigurations here can allow unauthorized traffic to traverse the cluster.
- kube-proxy – Routing traffic to the correct pods and enabling port forwarding for user access to applications. Misconfigurations can lead to unauthorized access, traffic manipulation, or unintended data leaks.
Now that we’ve explored the key Kubernetes components, their roles, and the key risks associated with them, you should have a solid understanding of the core architecture. This foundation will serve as a reference point for the upcoming articles as we dive deeper into Kubernetes security and best practices.
To learn more about our Kubernetes offering: https://www.o3c.no/services/kubernetes-security-assessment