K8s: remove friction and reduce

frustration with these tricks

A Collection of tricks and tools to Solve All Your Debugging Needs

@RajhiSaifeddine

bento.me/saifeddinerajhi
# whoami

● Saifeddine Rajhi
(@RajhiSaifeddine,
bento.me/saifeddinerajhi)
● Sr. Platform engineer
1. Running a local cluster
There are so many options:

CoLiMa, Docker Desktop, k3d, kind, minikube, orbstack, Rancher Desktop…

2. Switch between namespaces or clusters

kubectx & kubens!

kubectl create namespace demo

kubens demo

Tip #1: alias/rename them to "kns" and "kctx"

Tip #2: add kubectl-fzf for extra convenience

Tip #3: "kns -" switches back to the previous namespace

3. Waiting for a Deployment to be ready

kubectl create deployment demo --image=nginx

kubectl wait deployment demo --for=condition=Available

🤔 Conditions?
"kubectl wait" can wait on any condition. See:

kubectl get deployments -o 'jsonpath={.items[0].status.conditions[*].type}{"\n"}'

# Wait for pod "busybox" to be Ready

Kubectl run pod busybox –image=busybox
kubectl wait --for='jsonpath={.status.conditions[?(@.type=="Ready")].status}=True' pod/busybox

Many things have conditions! Including: Deployments, Pods, Nodes, Jobs, ReplicaSets, StatefulSets, DaemonSets,
… and CRDs too!
4. Turn a Deployment off and on again
kubectl rollout restart deployment hello

How does it really work?

kubectl get deployment hello -o jsonpath={.spec.template.metadata.annotations}

{"kubectl.kubernetes.io/restartedAt":"2023-10-09T09:51:25+02:00"}

It's "just" an annotation on the Pod template, triggering a rolling update

(exactly as if we had changed e.g. the image).

It also works on Statefulsets and Daemonsets.

5. Really turn a Deployment off and on again

Scale to zero:

kubectl scale deployment demo --replicas=0

And back up:

kubectl scale deployment demo --replicas=1

6. Connect to a Service in a different Namespace

kubectl create namespace blue

kubectl create --namespace blue deployment yellow --image jpetazzo/color

kubectl expose --namespace blue deployment yellow --port 80

kubectl create service externalname yinb \

--external-name yellow.blue.svc.cluster.local

kubectl run test --rm --restart=Never --attach \

--image fedora curl yinb
7. Generate a YAML manifest

NO DOCS

NO EXTERNAL WEBSITES

NO CHATGPT or AI

(e.g. for CKA/CKAD/CKS😁)

kubectl create deployment demo --image=nginx -o yaml --dry-run=client

This works with kubectl run/create/expose!

8. kubectl patch all the things

kubectl patch deployment demo --patch "

spec:
template:
spec:
containers:
- name: smaller-nginx
image: nginx:alpine
"
9. Label columns
The --label-columns flag allows you to specify which labels associated with the nodes you
want to display in the output.

kubectl get nodes --label-columns \

kubernetes.io/arch,node.kubernetes.io/instance-type,topology.kubernetes.io/zone

NAME STATUS ROLES AGE VERSION ARCH INSTANCE-TYPE ZONE

scw-arm-...3b Ready <none> 5d v1.28.0 arm64 AMP2-C4 fr-
par-2
scw-arm-...b5 Ready <none> 6d v1.28.0 arm64 AMP2-C4 fr-par-
2
scw-arm-...49 Ready <none> 6d v1.28.0 arm64 AMP2-C4 fr-par-
2
10. Find the JSON path of something

gron to the rescue!

Gron makes JSON greppable!

kubectl get nodes -o json | gron | grep -i pressure

json.items[0].status.conditions[0].type = "MemoryPressure";
json.items[0].status.conditions[1].message = "kubelet has no disk pressure";
json.items[0].status.conditions[1].reason = "KubeletHasNoDiskPressure";
json.items[0].status.conditions[1].type = "DiskPressure";

=>> See also "gron --ungron" for extra funsies!

11. Get an image with almost any tool you want

"Hey I need an image with terraform, kubectl, helm, curl, and ffmpeg!"

"Sure, let's write a Dockerfile, and…"

docker run -ti nixery.dev/shell/terraform/kubectl/kubernetes-helm/curl/ffmpeg

DONE

(Image gets generated on the fly;

this is probably the magickest thing you'll see today)
infuser
infuser is built around tea CLI which automatically installs any tool that you invoke. Let's say you run
curl in a container, but it's not installed. No problem, tea will first install it and then invoke your
command:

docker run --rm -it ghcr.io/teaxyz/infuser

...

# curl is not installed...

tea $ curl https://google.com

# installed: ~/.tea/curl.se/ca-certs/v2023.5.30
# installed: ~/.tea/openssl.org/v1.1.1u

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">

<H1>301 Moved</H1>
netshoot

if you need to troubleshoot anything network-related, you can always reach for
netshoot. It contains all the networking tools you might need, all you need to do is run:
docker run --rm -it nicolaka/netshoot
# OR
kubectl run tmp-shell --rm -i --tty --image nicolaka/netshoot
netshoot

if you need to troubleshoot anything network-related, you can always reach for
netshoot. It contains all the networking tools you might need, all you need to do is run:
docker run --rm -it nicolaka/netshoot
# OR
kubectl run tmp-shell --rm -i --tty --image nicolaka/netshoot
12 - kubectl debug

You can instead add a debugging container using kubectl debug.

If you specify the -i/--interactive argument, kubectl will automatically attach to the
console of the Ephemeral Container.

kubectl run ephemeral-demo --image=registry.k8s.io/pause:3.1 --restart=Never

kubectl debug -it ephemeral-demo --image=busybox:1.28 --target=ephemeral-demo

13 - kubectl plugins A.K.A Krew

Install Krew

Run kubectl krew install <PLUGIN_NAME> to install a plugin via Krew.

kubectl krew search

 14 - Use AI to debug Kubernetes
Kubernetes & AI team-up

Demo: https://github.com/k8sgpt-ai/k8sgpt/blob/main/images/demo4.gif
Kubectl OpenAI plugin ✨:

This project is a kubectl plugin to generate and apply Kubernetes manifests using OpenAI

GPT.

Demo: https://asciinema.org/a/MEXrlAqUjo7DMnfoyQearpVQ7
Thank you !!