Using Kubernetes Ephemeral Containers for Troubleshooting

A time lapse picture of blue and red streaks of light
  • To have a smaller attack vector area.
  • To have faster-scanning performance.
  • Reduced image size.
  • To have a faster build and CD/CI cycle.
  • To have fewer dependencies.

Configuration of Ephemeral Containers

  • They are not to be restarted.
  • Resources definition is not allowed.
  • Ports are not allowed.
  • Startup, liveness, and readiness probes are not allowed.

Enabling Ephemeral Containers in Your Cluster

$ kubectl debug -it <POD_NAME> --image=busybox
Defaulting debug container name to debugger-wg54p.
error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").

Using Ephemeral Containers

$ kubectl create deployment nginx-deployment --image=nginx
deployment.apps/nginx-deployment created
$ kubectl get podsNAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-66b6c48dd5-frsv9 1/1 Running 6 62d
$ kubectl debug -it pods/nginx-deployment-66b6c48dd5-frsv9 --image=busyboxDefaulting debug container name to debugger-r44v5.
If you don't see a command prompt, try pressing enter.
/ #
/ # ping ( 56 data bytes
64 bytes from seq=0 ttl=112 time=9.797 ms
64 bytes from seq=1 ttl=112 time=9.809 ms
/ # nc --helpBusyBox v1.34.1 (2021-11-11 01:55:05 UTC) multi-call binary.

Usage: nc [OPTIONS] HOST PORT - connect
nc [OPTIONS] -l -p PORT [HOST] [PORT] - listen
$ kubectl describe pods nginx-deployment-66b6c48dd5-frsv9Name:         nginx-deployment-66b6c48dd5-frsv9
Namespace: default
Priority: 0
Node: node1/x.x.x.x
Start Time: Mon, 30 Aug 2021 21:50:17 +0200
Labels: app=nginx
Annotations: <none>
Status: Running
Controlled By: ReplicaSet/nginx-deployment-66b6c48dd5
Container ID: containerd://6367af3713afb85ecb1e1a057ba9db4e3b2c48f39fee6a248cd2811e198001aa
Image: nginx:1.14.2
Ephemeral Containers:
Container ID: containerd://eec23aa9ee63d96b82970bb947b29cbacc30685bbc3418ba840dee109f871bf0
Image: busybox
Image ID:
Port: <none>
Host Port: <none>
State: Running
Started: Mon, 15 Nov 2021 20:28:57 +0100
Ready: False
Restart Count: 0
Environment: <none>
Mounts: <none>

Process Namespace Sharing with Ephemeral Containers

$ kubectl debug -it nginx-deployment-66b6c48dd5-frsv9 --image=busybox --share-processes --copy-to=debug-pod
1 root 0:00 /pause
6 root 0:00 nginx: master process nginx -g daemon off;
11 101 0:00 nginx: worker process
12 root 0:00 sh
17 root 0:00 ps aux
# ls /proc/6/root/etc/nginx
conf.d koi-utf mime.types nginx.conf uwsgi_params fastcgi_params koi-win modules scgi_params win-utf


Further reading




>> << Build Your Internal Kubernetes Platform With Virtual Clusters, Namespace Self-Service & Secure Multi-Tenancy

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

How I hacked Agile to work for DevOps teams

Applause crowd testing platform: Testing in the Sprint

Kotlin Enum Classes

Vue.js + Flask + Flask-SQLAlchemy: Part 3: deployment on DigitalOcean

5 Ways to Access Kubernetes Clusters

Overhead photo of a multi-lane freeway


Learn Lazy Loading Through a Simple Example

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Loft Labs

Loft Labs

>> << Build Your Internal Kubernetes Platform With Virtual Clusters, Namespace Self-Service & Secure Multi-Tenancy

More from Medium

Disaster recovery in the Kubernetes cluster.

What’s new in MicroK8s v1.23?

Sequencing Container Startup

Untangling Reclaim Policies for Persistent Volume in Kubernetes