NGINX Modern Apps > Class 12 - The Path to Understanding Kubernetes and Containers > Module 3 - Kubernetes Troubleshooting Source | Edit on
Kubernetes Troubleshooting¶
Throughout this class you have used several first step methods in troubleshooting. Using get and describe help start the process, but what if they do not provide enough detail to lead you to the problem? In this lab we’ll show you some additional commands you can use to get information or configure all the objects in your cluster.
Explain¶
Let’s start with the explain command. The Kubernetes explain command provides documentation of the resource or specific field of the resource. In past labs, you used imperative commands to create pods but we only covered the fields used. You can find reference to all objects right from the cli.
Example:
kubectl explain <resource>
Now, let’s walk some parts of the pod resource. This command will show the fields available to you to configure a pod.
kubectl explain pod
Digging deeper, what details are available under metadata?
kubectl explain pod.metadata
As you can see from the output, the name field is required when declaring a pod.
Get¶
Kubernetes get command will fetch the status and name of an object. In the below example, instead of requesting information on a specific object we are seeking information on all objects within the test namespace.
kubectl get all -n test
Describe¶
Kubernetes describe shows details of a specific resource or group of resources. This command joins many API calls together to form a detailed description of a given resource or group of resources. This is the command you have run several times during this class to find detailed information on your deployed resources.
kubectl describe <resource_type> <resource_name> -n <namespace>
Events¶
Kubernetes events can provide valuable insights to events from controllers, schedulers, pods and nodes. You can, and should, filter down events.
kubectl get events -n test
Get live events:
kubectl get events -n test --watch
Filter events by namespace and resource type.
kubectl get events -n test --field-selector involvedObject.kind=Pod
Filter events by namespace, resource type, and pod name.
kubectl get events -n test --field-selector involvedObject.kind=Pod --field-selector involvedObject.name=testpod
To sort events by time you can use the below command. The –sort-by command is actually reading the Kubernetes JSON returned data to extract the lastTimestamp field.
kubectl get events -n test --sort-by={.lastTimestamp}
Logs¶
The logs command allows you to view logs generated by a pod. You have a run a similar command in the Container lab. In these two steps you’ll view logs in a single pod and all pod logs from the deployment.
kubectl logs testpod -n test
Deployment logs:
kubectl logs deploy/lab-deploy -n test
Execute¶
You can connect to the shell of a running container by using the below command.
kubectl exec -it testpod -n test -- /bin/bash
You should now see a prompt:
lab@k3s-leader:~$ kubectl exec -it testpod -n test -- /bin/bash
root@testpod:/#
Feel free to run some Linux commands such as
- pwd
- ls -la
To exit the shell, type exit
But you don’t have to access the shell to run your commands, you can pass the command to the shell.
kubectl exec -it testpod -n test -- ls -la
If your pod has more than one container you must specify the container you want to connect to with the -c flag as in the example below.
kubectl exec -it <pod_name> -c <container_name> -n <namespace> -- /bin/bash
DNS Utils¶
For this next troubleshooting exercise, you’ll deploy a special dnsutils container image. This container has dnsutils installed and will allow you to view how services are registered in CoreDNS.
kubectl run dnsutils --image=registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3 --restart=Always -n test -- /bin/bash -c "sleep infinity"
Once deployed and running, you can execute dig commands from inside the cluster using the dnsutils tools. If this command fails, wait a few seconds then execute again.
kubectl exec -it dnsutils -n test -- dig lab-deploy-svc.test.svc.cluster.local