Introduction
This post contains random notes related to hacking of Kubernetes environments.
Mayhaps it won’t make any sense, dear reader - but it will at least help me remember.
This is a “living document” (cringe) - hopefully I will update it if I find new ways toward cluster admin.
Use your powers for “samhällsnytta” and what not.
“How the heck did X really work?”
- “Kubernetes Deconstructed” by Carsen Andersen
- Kubernetes The Hard Way
- “Effective RBAC” by Jordan Liggitt
- Official documentation - Concepts
- Official documentation - Authentication
- Official documentation - Authorization
- Official documentation - Kubelet authentication and authorization
- Official documentation - Kubelet TLS bootstrapping
- Official documentation - PSP
- Official documentation - Network policy
- Official documentation - Service catalog
Security resources
- “Kubernetes Security” website
- Kubernetes Security Best Practices
- CIS Benchmark
- Hardening K8S from scratch
- 4ARMED’s blog posts
Tools
- kubectl
- etcdctl
- auger - Encode/Decode K8S data stored in etcd
- helm - Package manager
- kube-hunter - Offensive testing tool
- kube-bench - Automated CIS benchmark
- kubletmein - Exploitation of node bootstrapping (and cloud meta-data)
- Mikael Kall’s Docker tools
- Fully stocked Docker image
- A shipload of static Linux x86_64 binaries!
- [rbac-lookup - RBAC enumeration tool](https://github.com/reactiveops/rbac-lookup.git tools/rbac-lookup)
- [rakkess - RBAC enumeration tool](https://github.com/corneliusweig/rakkess.git tools/rakkess)
- [kubectl-who-can - RBAC enumeration tool](https://github.com/aquasecurity/kubectl-who-can.git tools/kubectl-who-can)
- [evilchart - Helm chart used for privilege escalation](https://github.com/4ARMED/evilchart.git tools/evilchart)
Reconnaissance
Ports
| Port | Description | Notes |
|---|---|---|
| 2379-2380/TCP | etcd | Backing store for all cluster data. Pwnage == “system:masters” |
| 4194/TCP | cAdvisor metrics | Information disclosure about nodes/pods, potential DoS factor |
| 6443/TCP | Master API port | Even if restricted by authN, risky exposure |
| 8080/TCP | Insecure master API port | Fun for the whole family! |
| 10250/TCP | Kublet API - RW | Information disclosure and/or total carnage |
| 10255/TCP | Kublet API - RO | Informaton disclosure |
| 10256/TCP | kube-proxy health | Meh. |
| 44134-44135/TCP | Tiller | Service used by Helm package manager, no authentication by default |
| 30000-32767/TCP | Default NodePort range | Services directly exposed via NodePort type |
DNS
TBD
Authenticated discovery
$ DATAPOINTS="$ kubectl api-resources -o name"
$ for DATAPOINT in ${DATAPOINTS}; do echo "${DATAPOINT}"; kubectl describe "${DATAPOINT}" &> "results/namespace-${DATAPOINT}"; kubectl describe "${DATAPOINT}" -A &> "results/global-${DATAPOINT}"; done
Exploitation
Relevant vulns / CVEs
| Title | Component | Description |
|---|---|---|
| CVE details vendor page | N/A | List of publicly disclosed vulnerabilities |
| CVE-2019-5736 PoC | Docker runtime | Container escape |
| CVE-2018-1002105 exploit | API server | Privilege escalation |
| CVE-2019-5021 write-up | Alpine image | Null password for root account |
Loot
Interesting files in containers:
| Path | Description |
|---|---|
| /var/run/secrets/ | Injected K8S secrets, such as credentials and other juicy stuff. Usually at least a service token |
| /etc/resolv.conf | Discover search domains, useful for further enumeration |
Interesting files in control plane:
| Path | Description |
|---|---|
| … | … |
Kubelet API
Query pods on node:
$ BEARER="Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
$ http --verify=no GET 'https://10.13.37.42:10250/pods/' "${BEARER}"
Execute command in container:
$ BEARER="Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
$ http --form --verify=no POST 'https://10.13.37.42:10250/run/<NAMESPACE>/<POD NAME>/<CONTAINER NAME>' "${BEARER}" 'cmd=whoami'
Random notes
Insufficient access control for etcd
It is not uncommon to utilize the same etcd cluster for both the control plane and other services, such as Calico networking.
If access control is not properly configured,
a node/application may be able to access and manipulate control plane data.