How to TCPdump effectively in Kubernetes (part 1)

📅 February 3, 2022 ✍️ Philippe Bogaerts ⏱️ 2 min read 📁 Kubernetes

tcpdump Kubernetes Container

In a previous blog post, we focused on how to TCPdump in docker containers (see https://xxradar.medium.com/how-to-tcpdump-effectively-in-docker-2ed0a09b5406).

Although the information is still very useful and valid for troubleshooting K8S pods, it might get more difficult figuring out which containers to attach to on what node, etc... but it is still a very valid approach.

While focusing on an easier way, I came across the command:

kubectl patch

This command allows to update an existing deployment for example. It basically does the trick outlined in the previous post but fully automatic.

So if you have deployment running like for example:

kubectl create -f https://raw.githubusercontent.com/xxradar/kuberneteslearning/master/radarhack-deploy.yaml

kubectl create -f https://raw.githubusercontent.com/xxradar/kuberneteslearning/master/radarhack-expose-clusterip.yaml

You should be able to access it on:

# kubectl get services
...
my-radarhack-clusterip   ClusterIP   10.104.201.226   <none>   80/TCP   36d

# curl http://10.104.201.226/
<HTML>
<HEAD>
<TITLE>RADARHACK.COM by XXRADAR</TITLE>
...

So far so good. Let's focus on how to add the TCPdump container to the deployment. Create following file ex. patch.yaml:

spec:
  template:
    spec:
      containers:
      - name: tcpdumper
        image: docker.io/dockersec/tcpdump

And apply it:

kubectl patch deployment radarhack-deployment --patch "$(cat patch.yaml)"

You should be able to see that the TCPdump container is automatically added to the pods (please note that the pods are recreated, which is not exactly the same as in the previous blogpost, where you connect to a running pod/container):

# kubectl get deployment radarhack-deployment --output yaml
apiVersion: extensions/v1beta1
kind: Deployment
...
    spec:
      containers:
      - image: docker.io/dockersec/tcpdump
        imagePullPolicy: Always
        name: tcpdumper
        resources: {}
      - image: docker.io/xxradar/naxsi5
        imagePullPolicy: Always
        name: radarhack
        ports:
        - containerPort: 80
          protocol: TCP
...

Now you can attach to the pod (and if traffic is generated):

# kubectl get pod
NAME                                   READY   STATUS    RESTARTS   AGE
radarhack-deployment-7c6b8f595-85dkt   3/3     Running   0          33m
radarhack-deployment-7c6b8f595-8qffp   3/3     Running   0          33m
radarhack-deployment-7c6b8f595-grdxz   3/3     Running   0          33m

# kubectl attach -it radarhack-deployment-7c6b8f595-85dkt
Defaulting container name to tcpdumper.
Use 'kubectl describe pod/ -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
08:14:38.740879 IP 10.244.0.0.48528 > radarhack-deployment-7c6b8f595-85dkt.80: Flags
[.], ack 7945, win 360, options [nop,nop,TS val 3777667637 ecr 3777658557], length 0
08:14:38.936447 ARP, Request who-has radarhack-deployment-7c6b8f595-85dkt tell
10.244.2.1, length 28
08:14:38.936473 ARP, Reply radarhack-deployment-7c6b8f595-85dkt is-at fa:7a:24:85:f1:56
(oui Unknown), length 28
...