Debug API

Overview

The Cluster Wide Controller (CWC) debug API enables running diagnostic commands from the local workstation, targetting specified TMM Pods. This method simplifies the troubleshooting workflow, removing the need to first log in to specific debug containers prior to run diagnostic commands.

This document guides you through understanding and running the CWC debug API.

Diagnostic utilities

The CWC API supports the following diagnostic utilities:

Utility	Description
ping	Used to perform a ping for the specified IPv4 address.
ping6	Used to perform a ping for the specified IPv6 address.
traceroute	Used to perform a traceroute to the specified IPv4 address.
traceroute6	Used to perform a traceroute to the specified IPv6 address.
tmctl	Used to retrieve a table from tmctl with the corresponding statistics (ex: virtual_server_stat)
tcpdump	Used to perform a tcpdump with the specified filters. Command will timeout after 30 seconds.
configview	Used to perform a configview query either using UUID or CRD type. Send configview --help for usage.
mrfdb	Used to perform an mrfdb query. Send mrfdb --help for usage. NOTE: the equals sign between flags and values is not supported. Use spaces to separate arguments.
lsndb	Used to perform an lsndb query. Send lsndb with no arguments for usage.

Alternate namespaces

When the CNFs CWC controller.f5_lic_helper.cwcNamespace parameter is set to a namespace other than default, the debug.cwcNamespace parameter must be set to the same value in the BIG-IP Controller Helm values file. In the example below, the parameter is set to cnf-telemetry.

debug:
  rabbitmqNamespace: "cnf-telemetry"

CWC Debug REST APIs

The sections below describe the CWC REST APIs. Use the Procedure section to see how the CWC API can be used to run a diagnostic command.

POST Request

Use the POST request to create new job IDs. You can request the CWC to run a utility on a debug container by POSTing a request using /debug API.

POST request CWC API Endpoint

https://f5-cnf-cwc.cnf-telemetry:30881/debug 

POST request Body

{
 namespace: <kubernetes namespace>, 
 command: <ping/ping6/traceroute/trceroute6/tmctl/tcpdump>, 
 params: <arguments to the utility>, 
 tmm-id: <TMM in which the utility is to be run> 
}

Command examples

The commands below are run in the same directory as the cert-gen directory created when setting up the CNFs CWC. i

PING

curl --request POST https://f5-cnf-cwc.cnf-telemetry:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"ping\",\"params\":\"10.44.0.6\",\"tmmId\":\"f5-tmm-6cdbc6bb65-j2r7d\"}"

TRACEROUTE

curl --request POST https://f5-cnf-cwc.cnf-telemetry:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"traceroute\",\"params\":\"10.44.0.6\",\"tmmId\":\"\f5-tmm-6cdbc6bb65-j2r7d"}

TCPDUMP

curl --request POST https://f5-cnf-cwc.cnf-telemetry:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"tcpdump\",\"params\":\"-nni,internal,-c,4\",\"tmmId\":\"f5-tmm-6cdbc6bb65-j2r7d\"}

TMCTL

curl --request POST https://f5-cnf-cwc.cnf-telemetry:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"tmctl\",\"params\":\"virtual_server_stat\",\"tmmId\":\"f5-tmm-6cdbc6bb65-j2r7d\"}"

CONFIGVIEW

Configview can be triggered either by UUID or CRD type.

Using UUID:

Note: Use the following command to obtain UUIDs from TMM: oc logs deploy/f5-tmm -c f5-tmm | grep -i uuid.

curl --request POST https://f5-spk-cwc.default:30881/debug \
--cert api-server-secrets/ssl/client/certs/client_certificate.pem  \
--key api-server-secrets/ssl/client/secrets/client_key.pem \
--cacert api-server-secrets/ssl/ca/certs/ca_certificate.pem \
--data-raw "{\"namespace\": \"sagnihotri\",\"command\":\"configview\",\"params\":\"uuid sagnihotri-f5ing-vlan3-vlan -s tmm0:8850\",\"tmmId\":\"f5-tmm-7bb9bc4994-8t492\"}"

Using CRD:

curl --request POST https://f5-spk-cwc.default:30881/debug \
--cert api-server-secrets/ssl/client/certs/client_certificate.pem \
--key api-server-secrets/ssl/client/secrets/client_key.pem \
--cacert api-server-secrets/ssl/ca/certs/ca_certificate.pem \
--data-raw '{"namespace": "sagnihotri","command":"configview","params":"f5-big-net-vlans --name f5ing-vlan1 -n sushma-alpha","tmmId":"f5-tmm-7bb9bc4994-8t492"}'

Available objects (CRD) for viewing include:

• f5-spk-vlan
• virtual-server

MRFDB

curl --request POST https://f5-cnf-cwc.default:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"mrfdb\",\"params\":\"-ipport memcachedhost:11211 -serverName server -displayAllBins\",\"tmmId\":\"f5-tmm-6cdbc6bb65-j2r7d\"}"

LSNDB

curl --request POST https://f5-cnf-cwc.default:30881/debug \
--cert cwc_api/client_certificate.pem  \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem \
--data-raw "{\"namespace\": \"cnf-gateway\",\"command\":\"configview\",\"params\":\"list all\",\"tmmId\":\"f5-tmm-6cdbc6bb65-j2r7d\"}"

GET Requests

Use the Debug API to run a utility, and add jobs to the queue. The job handler routine handles jobs by sending the request to the designated debug sidecar and waits for its reply. The job goes through its life cycle and comes to completion when the task is successfully executed by debug sidecar and replies the response to CWC.

Note: You can create a maximum of 5 jobs, each lasting for up to 6 minutes.

To retrieve the required Job ID response, Jobs list, and TMMs list details from the debug sidecar to CWC, use GET Request.

GET API Request for getting the list of TMMs in a specified namespace

Can query to get the list of TMMs in a specified namespace as shown below:

curl -s https://f5-cnf-cwc.cnf-telemetry:30881/debug/<namespace>/tmms \
--cert cwc_api/client_certificate.pem \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem

The list of TMMs are displayed with their Running status.

GET API Request for getting the Job ID Response

Can query to get the Job ID response to the completed job as shown below:

curl https://f5-cnf-cwc.cnf-telemetry:30881/debug/<JOB ID> \
--cert cwc_api/client_certificate.pem \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem

GET API Request for getting the list of Jobs

Can query to get the list of Jobs as shown below:

curl https://f5-cnf-cwc.cnf-telemetry:30881/debug/jobs \
--cert cwc_api/client_certificate.pem \
--key cwc_api/client_key.pem \
--cacert cwc_api/ca_certificate.pem

The list of jobs are displayed with status (In Progress / Complete) and description of the commands.

Requirements

Ensure you have:

• Installed the CNFs software.
• Installed the CNFs CWC.

Procedure

Use this procedure to ping a remote host from the debug sidecar using the CWC Debug API.

1. Change into the directory with the CNFs Software files.

2. As described in the CNFs Licensing guide, create a new directory for the CWC REST API certificates:

   mkdir cwc_api
   

3. Copy each of the certificates into the new directory:

   cp api-server-secrets/ssl/client/certs/client_certificate.pem cwc_api
   

   cp api-server-secrets/ssl/ca/certs/ca_certificate.pem cwc_api
   

   cp api-server-secrets/ssl/client/secrets/client_key.pem cwc_api
   

4. Obtain the name of the TMM Pod(s) in the Project:

   In this example, the CWC is in the cnf-telemetry namespace, and the TMM Pod(s) are in the cnf-gateway namespace.

   curl -s https://f5-cnf-cwc.cnf-telemetry:30881/debug/cnf-gateway/tmms \
   --cert cwc_api/client_certificate.pem  \
   --key cwc_api/client_key.pem \
   --cacert cwc_api/ca_certificate.pem
   

   In this examle, the TMM Pod name is f5-tmm-595985589b-shxx2.

   TMM ID: f5-tmm-595985589b-shxx2	 STATUS: Running
   

5. Use the TMM Pod name to create the ping diagnostic job:

   The following example shows the POST request query using curl:

   curl --request POST https://f5-cnf-cwc.cnf-telemetry:30881/debug \
   --cert cwc_api/client_certificate.pem  \
   --key cwc_api/client_key.pem \
   --cacert cwc_api/ca_certificate.pem \
   --data-raw '{"namespace":"cnf-gateway","command":"ping","params":"192.168.10.10","tmmId":"f5-tmm-595985589b-shxx2"}'
   

   In this example, the job ID is 4625993b-31e2-4570-8b41-2c1296026c16.

   JobID: 4625993b-31e2-4570-8b41-2c1296026c16
   

6. Query the list of available jobs:

   Note: There can be a maximum of 5 jobs, each lasting up to 6 minutes.

   curl -s https://f5-cnf-cwc.cnf-telemetry:30881/debug/jobs \
   --cert cwc_api/client_certificate.pem  \
   --key cwc_api/client_key.pem \
   --cacert cwc_api/ca_certificate.pem \
   

   In the example below, Ping is the most recent job added to the queue.

   JobId: 5775e5e2-1c26-43c9-bf4d-50a87a6ae188	Status: Complete	Desc: Running Tmctl
   JobId: 13ec0601-366e-45ff-859b-6c51b321ce84	Status: Complete	Desc: Running Traceroute
   JobId: e1fba37b-0a6d-4e16-ae69-db86244f1721	Status: Complete	Desc: Running Tcpdump
   JobId: 4625993b-31e2-4570-8b41-2c1296026c16	Status: Complete	Desc: Running Ping
   

7. Use the job ID to run the ping diagnostic:

   curl https://f5-cnf-cwc.cnf-telemetry:30881/debug/4625993b-31e2-4570-8b41-2c1296026c16 \
   --cert cwc_api/client_certificate.pem  \
   --key cwc_api/client_key.pem \
   --cacert cwc_api/ca_certificate.pem \
   

   Job ID: 4625993b-31e2-4570-8b41-2c1296026c16
   Namespace: cnf-gateway
   Status: Complete ... 1/1 responses received
   Pending Responses: None
   Start Time: October/18/2022 - 17:54:55
   
   Last Updated: October/18/2022 - 17:54:55
   
   End Time: October/18/2022 - 17:54:55
   
   TmmID: f5-tmm-595985589b-shxx2
   Tmm Job Data: ping 192.168.10.10
   PING 192.168.10.10 (192.168.10.10) 56(84) bytes of data.
   64 bytes from 192.168.10.10: icmp_seq=1 ttl=255 time=2.40 ms
   64 bytes from 192.168.10.10: icmp_seq=2 ttl=255 time=6.09 ms
   64 bytes from 192.168.10.10: icmp_seq=3 ttl=255 time=2.17 ms
   64 bytes from 192.168.10.10: icmp_seq=4 ttl=255 time=1.42 ms
   

Feedback

Provide feedback to improve this document by emailing cnfdocs@f5.com.