F5 BIG-IP SSL Orchestrator Training Lab > SSLO 101: Essential SSL Visibility with SSL Orchestrator (Agility 2022 | 2 hours) Source | Edit on
9. Appendix 1 - Common Testing Commands¶
The following are some simple, but powerful commands that are useful in developing and troubleshooting SSL visibility projects.
9.1. View TLS/SSL certificates using a command line tool¶
The OpenSSL tool can be used to establish a TLS session and display the certificate returned.
openssl s_client -showcerts -connect www.google.com:443
9.2. Packet capture¶
Second only to debug logging, packet captures are crucial to troubleshooting (and simply validating) network and service-related issues. Each security service is connected to separate "ingress" and "egress" VLANs, traffic going into the service from the F5, and traffic leaving the service back to the F5, respectively. To verify that traffic is entering, or leaving a security device, insert a tcpdump "tap" on the appropriate VLAN.
tcpdump -lnni [VLAN]:nnn -Xs0
Note that the service VLANs reside within application service containers, so must be referenced accordingly. The easiest way to derive this path is from the BIG-IP UI. Navigate to the Network - VLANs menu. In the Partition / Path column for the desired VLAN, copy the path beyond the Common/ string. For example, if the path is "Common/ssloN_IPS_in.app", copy "ssloN_IPS_in.app". The full path will be this value, plus the same string again without the ".app" extension. The VLAN path will therefore look like this:
From the BIG-IP command line, insert the tcpdump tap on the ingress side of this IPS service like this:
tcpdump -lnni ssloN_IPS_in.app/ssloN_IPS_in
Pass traffic through SSLO to verify that data is flowing to the inline service. Switch the VLAN value to the egress VLAN to also verify data is flowing out of the inline service. To view decrypted traffic, add "-Xs0" (zero) to the tcpdump command.
tcpdump -lnni ssloN_IPS_in.app/ssloN_IPS_in -Xs0
And to filter out ICMP traffic and other unneeded flows, add filters to the end of the capture.
tcpdump -lnni ssloN_IPS_in.app/ssloN_IPS_in not icmp
9.3. Control the SSL Forward Proxy certificate cache¶
The behavior of the SSL Forward Proxy changes after a certificate is cached, which will make it difficult to troubleshoot some issues. The following allows you to both list and delete the certificates in the cache.
tmsh show ltm clientssl-proxy cached-certs clientssl-profile [CLIENTSSL PROFILE] virtual [INGRESS TCP VIP] tmsh delete ltm clientssl-proxy cached-certs clientssl-profile [CLIENTSSL PROFILE] virtual [INGRESS TCP VIP]
9.4. Isolate SSLO traffic¶
Any given website will be full of images, scripts, style sheets, and very often references to document objects on other sites (like a CDN). This can make troubleshooting very complex. The following cURL commands allow you to isolate traffic to a single request and response.
curl -vk https://www.bing.com curl -vk --proxy 10.30.0.150:3128 https://www.bing.com curl -vk --proxy 10.30.0.150:3128 --location https://www.bing.com
Optionally, between each cURL test, delete the certificate cache and start logging:
tmsh delete ltm clientssl-proxy cached-certs clientssl-profile [CLIENTSSL PROFILE] virtual [INGRESS TCP VIP] && tail -f /var/log/apm
There is simply nothing better than debug logging for troubleshooting SSL intercept issues. The SSL Orchestrator in debug mode pumps out an enormous set of logs, describing every step along a connection's path. Remember to never leave debug logging enabled.
tail -f -n 0 /var/log/apm
9.6. SSL inspection¶
ssldump -AdNd -i [VLAN] port 443 <and additional filters> tcpdump -i 0.0:nnn -nn -Xs0 -vv -w <file.pcap> <and additional filters> ssldump -nr <file.pcap> -H -S crypto > text-file.txt
TLS is rarely the issue, but a sight or configuration error may render some sites inaccessible.
9.7. Control the URL Filtering database¶
To show the current status of the database:
tmsh list sys url-db download-result
To initiate (force) the URL DB to update:
tmsh modify sys url-db download-schedule all status true download-now true
To verify that the URL DB is actively updating:
tcpdump -lnni 0.0 port 80 and host 22.214.171.124
9.8. External testing¶
Enter the site's address into the SSLLabs.com server test site at https://www.ssllabs.com/ssltest/ to see if the site has any unusual SSL/TLS requirements.