Troubleshoot Network Issues

Kubernetes is an orchestration tool for the docker container. Which make our life easy in terms of uptime of application, deployment and other things. But when it comes to networking, it becomes very difficult to find out the root cause.

This blog will be your next Arsenal for networking tools to help you to debug the issues in Kubernetes.

• QPERF
• Tcpdump

Let’s discuss the above tools one by one and understand the use case.

QPERF:

Qperf is an open-source tool to measure network performance. When you need to check the network performance between two Linux devices. Qperf helps us to measure both TCP and UDP protocols.

Qperf required a server and client. We need to run this utility in source and destination. Without any arguments, qperf is working like a server and destination we need to parse arguments like below

Run Qperf as server

qperf

Run qperf as a client for TCP

qperf -v {source server ip} tcp_bw tcp_lat

for UDP

qperf -v {source server ip} udp_bw udp_lat

TCPDUMP:

A great tool to capture traffic. I am working on the hybrid cloud where we are using multiple clouds and it's very difficult to troubleshoot network issues between two containers running in two different k8s clusters. Tcpdump help me trace the traffic and identify the root cause. Below are some common use case

Capture Incoming Get Request

tcpdump -i {interface} -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'

Capture Incoming Post Request

tcpdump -i {interface} -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504F5354'

Sometimes we face issues like some of the api calls not working in the hybrid infrastructure. Where we have connected hybrid infrastructure using IPSec. Because of MTU miss-matched, some API’s stop working. So we can set the correct MTU value to fix the same.

ifconfig <Interface_name> mtu <mtu_size> up