F5BigDohApp

Overview

The F5BigDohApp Custom Resource (CR) configures the Traffic Management Microkernel (TMM) to provide high-performance DNS resolution, caching and DNS64 translation mapping over secure HTTPS connections. The F5BigDohApp CR can also reference the F5BigIpsPolicy to intelligently protect applications from malignant network traffic, and the F5BigDnsCache CR to optimize DNS lookup performance with query caching.

This document guides you through understanding, configuring and installing a simple F5BigDohApp, and the optional F5BigDnsCache, F5BigIpsPolicy and F5BigLogProfile CRs.

CR parameters

The tables below describe the F5BigDohApp and F5BigDnsCache CR parameters used in this document.

F5BigDnsCache

The table below describes the F5BigDnsCache CR spec parameters used in this document. For the full list of parameters, refer to the F5BigDnsCache Reference.

_images/spk_info.png Note: DNS responses remain cached for the duration of the DNS record TTL.

Parameter Description
cacheType The DNS cache type: transparent.
transparent.localZones.name The Fully Qualified Domain Name for a localZone.
transparent.localZones.zoneType The zone type for the localZone: deny, refuse, static, transparent (default), type-transparent, or redirect.
transparent.localZones.records An array of records for this localZone.

F5BigDohApp

The table below describes the F5BigDohApp CR spec parameters used in this document. For the full list of parameters, refer to the F5BigDohApp Reference.

Parameter Description
destination.address Specifies the IPv4 address used by clients to resolve DNS queries.
destination.ipv6Address Specifies the IPv6 address used by clients to resolve DNS queries.
destination.port Specifies the service port used to resolve DNS queries. The default is 443.
logProfile Specifies the F5BigLogProfile to be used.
iRules Specifies one or more iRules CRs within F5BigDohApp CR.
serverIpProtocol Specifies the IP protocol for which you want the virtual server to direct traffic. Sample protocol names are tcp and udp.
dns.dnsCache Enables caching when referencing a F5BIGDnscache CR by metadata.name.
dnsOverHttps.name Specifies a unique name to identify the profile .
vlans.vlanList.item A reference to a F5BigNetVlan name.
clientSideHttp2.concurrentStreamsPerConnection Specifies the number of outstanding concurrent requests that are allowed on a single HTTP/2 connection. The default is 10.
clientSideHttp.maxHeaderSize Specifies the maximum header size. The dafault is 32768.
clientSideSsl.enableRenegotiation Enables/Disables Renegotiation support: true and false (default).
clientSideSsl.keyCertPairs.key References SSL/TLS private keys. Key names must be appended to the path file://etc/ssl/tls-keys-certs/.key.
clientSideSsl.keyCertPairs.cert References SSL/TLS certificates and intermediate CA certificates used to terminate secure ingress connections. Certificate names must be appended to the path file://etc/ssl/tls-keys-certs/.crt.
tcpSettings.clientSide Specifies a client side F5BigTcpSetting CR referenced by the virtual server, using the metadata.name parameter.
udpSettings.serverSide Specifies a server side F5BigUdpSetting CR referenced by the virtual server, using the metadata.name parameter.
pool.minActiveMembers Specifies the minimum number of members that must be available in one priority group.
pool.members Specifies a list of endpoint DNS servers used to resolve DNS queries.
pool.members.address Specifies an endpoint, or DNS server used to resolve DNS queries.
pool.members.port Specifies the endpoint service port used to resolve DNS queries. The default value is 53.
pool.members.priorityGroup Specifies the priority group for the pool member.
monitors.dns.enabled Enables monitoring the pool.members availability: true or false (default).
monitors.dns.queryName Specifies a fully qualified domain name the monitor sends in the DNS query probe.
monitors.dns.queryType Specifies the type of DNS query to send type that the monitor sends in DNS query probe: a (default) or aaaa.
monitors.dns.icmp.enabled Enables sending ICMP probes to verify the pool.members availability: true or false (default).
monitors.dns.recv The IP address that the monitor looks for in the DNS server response to the DNS query probe.

_images/spk_info.png Note: Refer to the Managing SSL/TLS certs and keys section prior to configuring the clientSideSsl parameters.

CR Examples

F5BigDnsCache

apiVersion: "k8s.f5net.com/v1"
kind: F5BigDnsCache
metadata:
  name: "cnf-dnscache"
  namespace: "cnf-gateway"
spec:
  cacheType: transparent
  transparent:
    localZones:
      - name: example.com
        zoneType: static
        records:
          - example.com. IN AAAA 2002::10:11:12:13

F5BigDohApp

apiVersion: "k8s.f5net.com/v1"
kind: F5BigDohApp
metadata:
  name: doh-app
spec:
  destination:
   address: "11.11.11.162"
   port: 443
  snat:
   type: "automap"
   iRules: ["dns-req"]
  serverIpProtocol: "udp"
  dns:
    dnsCache: "cnf-dnscache"
  dnsOverHttps:
    name: "doh-server"
  clientSideHttp2:
    concurrentStreamsPerConnection: 20
  clientSideHttp:
    maxHeaderSize: 32768
  clientSideSsl:
    enableRenegotiation: true
    keyCertPairs:
    - key: 'file://etc/ssl/tls-keys-certs/client.key'
      cert: 'file://etc/ssl/tls-keys-certs/client.crt'
  tcpSettings:
    clientSide: "tcp-profile-client"
  udpSettings:
    serverSide: "udp-profile-server"
  pool:
    minActiveMembers: 1
    members:
    - address: "10.244.99.110"
      port: 53
      priorityGroup: 1
  monitors:
    dns:
      enabled: true
      queryName: "webapp.net."
      queryType: "aaaa"
      recv: "2002::10:10:20:200"

CR shortName

CR shortNames provide an easy way to view installed CRs, and their configuration parameters. The CR shortName can also be used to delete the CR instance. The F5BigDohApp and F5BigDnsCache CR shortNames are dohapp and dnscache respectively.

View CR instance:

kubectl get dohapp -n <namespace>
kubectl get dnscache -n <namespace>

View CR configuration:

kubectl get dohapp -n <namespace> -o yaml
kubectl get dnscache -n <namespace> -o yaml

DNS Monitors

Prior to configuring and applying F5BigDohApp monitors to Service endpoints, it is important to understand the CR’s timeout and interval parameters, and their recommended configuration. The parameters behave as follows:

  • timeout is only observed when it is less than the interval: Endpoints are marked down when unanswered probes exceed the configured timeout.
  • timeout is not observed when it is greater than the interval: Endpoints are marked down when unanswered probes exceed the configured interval.

_images/spk_info.png Note: F5 recommends setting the timeout value to the same or less than the interval value.

Managing certs and keys

Read this section carefully to ensure the SSL/TLS certificates and keys referenced by the F5BigDohApp CR are encoded and installed into the cluster properly. These bullet points are essential:

  • When installing the BIG-IP Controller you must set tmm.tlsStore.enabled paramter to true.
  • The SSL/TLS certificates and keys must be Base64 encoded, and stored in a Secret named tls-keys-certs-secret.
  • TMM mounts the Secret named tls-keys-certs-secret to the file path file://etc/ssl/tls-keys-certs/.

_images/spk_warn.png Important: The tls-keys-certs-secret Secret must be created before the BIG-IP Controller is installed, otherwise the mount will fail and cause the TMM to enter a restart loop.

Use the steps below to generate a new SSL/TLS certficate and key, Base64 encode them, and then create the tls-keys-certs-secret Secret to store them in the cluster. F5 recommends using SSL/TLS certificates signed by a well-known certificate authority (CA) for production application traffic.

_images/spk_info.png Note: Use steps 4 - 6 if you already have an existing SSL/TLS certificate and key pair.

  1. Generate the CA signing certificate and key:

    openssl genrsa -out ca.key 4096

    openssl req -x509 -new -nodes -key ca.key -sha256 -days 365 -out ca.crt \
-subj "/C=US/ST=WA/L=Seattle/O=F5/OU=Dev/CN=ca"

  2. Generate the clientssl profile SSL/TLS certificate signing request (CSR):

    openssl genrsa -out client.key 4096

    openssl req -new -key client.key -out client.csr \
-subj "/C=US/ST=WA/L=Seattle/O=F5/OU=PD/CN=client.com"

  3. Sign the clientssl profile CSR with the CA:

    openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key \
-set_serial 101 -outform PEM -out client.crt -extensions req_ext -days 365 -sha256

  4. Base64 encode the SSL/TLS certificate and key:

    openssl base64 -A -in client.crt -out client-encode.crt
openssl base64 -A -in client.key -out client-encode.key

  5. Create the tls-keys-certs-secret Secret that stores the SSL/TLS certificate and key:

    echo "apiVersion: v1" > tls-keys-certs-secret.yaml
echo "kind: Secret" >> tls-keys-certs-secret.yaml
echo "metadata:" >> tls-keys-certs-secret.yaml
echo " name: tls-keys-certs-secret" >> tls-keys-certs-secret.yaml
echo "data:" >> tls-keys-certs-secret.yaml
echo -n " client.crt: " >> tls-keys-certs-secret.yaml
cat client-encode.crt >> tls-keys-certs-secret.yaml
echo " " >> tls-keys-certs-secret.yaml
echo -n " client.key: " >> tls-keys-certs-secret.yaml
cat client-encode.key >> tls-keys-certs-secret.yaml

  6. Install the Secret into the BIG-IP Controller Project:

    kubectl apply -f tls-keys-certs-secret.yaml -n cnf-gateway

Requirements

Ensure you have:

  • Installed a K8S Service object and application.
  • Installed the CNF Controller Pods.
  • A Linux based workstation.
  • Installed the dSSM Database to support persistence records.

Installation

Use the following steps to install the F5BigDohApp CR.

_images/spk_info.png Tip: Open a second shell to view the CNFs Event Logs while installing.

  1. Optional: To define the type of DNS and IPs events to capture, copy the F5BigLogProfile CR into a YAML file:

    apiVersion: "k8s.f5net.com/v1"
kind: F5BigLogProfile
metadata:
  name: "cnf-log-profile"
  namespace: "cnf-gateway"
spec:
  name: "dns-log"
  dns:
    enabled: true
    responseLogging: true
    queryId: true
  protocolInspection:
    enabled: true
    logPacket: true

  2. Install the F5BigLogProfile CR:

    oc apply -f cnf-log-cr.yaml

    In this example, the BIG-IP Controller logs indicate the F5BigLogProfile CR was added/updated:

    I0202 12:00:00.12348   1 event.go:282 Event(v1.ObjectReference{Kind:"F5LogProfile",
LogProfile cnf-gateway/cnf-log-profile was added/updated

  3. Optional: The example F5BigIpsPolicy CR rejects SOA record queries, and rejects dns_named_version_attempt and dns_os_solaris_exploit_sparc_overflow_attempt packet signatures. Copy and paste the CR into a YAML file:

    apiVersion: "k8s.f5net.com/v1"
kind: F5BigIpsPolicy
metadata:
  name: "cnf-ips"
  namespace: "cnf-gateway"
spec:
  services:
    - name: dns
      ports:
        - "53"
      compliances:
        - name: dns_disallowed_query_type
          valueType: string
          value: SOA
          action: reject
      signatures:
        - name: dns_named_version_attempt
          action: reject
        - name: dns_os_solaris_exploit_sparc_overflow_attempt
          action: reject

  4. Install the F5BigIpsPolicy CR:

    oc apply -f cnf-ips-policy.yaml

    In this example, the BIG-IP Controller logs indicate the F5BigIpsPolicy CR was added/updated:

    I0208 12:00:00.12345   1 event.go:282] Event(v1.ObjectReference{Kind:"F5ProtocolInspectionProfile", 
F5ProtocolInspectionProfile cnf-gateway/cnf-ips was added/updated

  5. Optional: Copy the F5BigDnsCache CR into a YAML file:

    In this example, the DNS cache creates an AAAA record, and returns authoritative DNS responses for the example.com domain.

    apiVersion: "k8s.f5net.com/v1"
kind: F5BigDnsCache
metadata:
  name: "cnf-dnscache"
  namespace: "cnf-gateway"
spec:
  cacheType: transparent
  transparent:
    localZones:
      - name: example.com
        zoneType: static
        records:
          - example.com. IN AAAA 2002::10:11:12:13

  6. Install the F5BigDnsCache CR:

    oc apply -f cnf-dnscache-cr.yaml

    In this example, the BIG-IP Controller logs indicate the F5BigDnsCache CR was added/updated:

    I0208 12:00:00.12345   1 event.go:282] Event(v1.ObjectReference{Kind:"F5Dnscache", 
F5Dnscache cnf-gateway/cnf-dnscache was added/updated

  7. Copy the F5BigDohApp into a YAML file:

    In the example below, clients can use 11.11.11.162 as their DNS resolver IP address.

    apiVersion: "k8s.f5net.com/v1"
kind: F5BigDohApp
metadata:
  name: "cnf-dohapp"
  namespace: "cnf-gateway"
spec:
  serverIpProtocol: "udp"
  destination:
   address: "11.11.11.162"
   port: 443
  snat:
   type: "automap"
   iRules: ["dns-req"]
  dns:
    dnsCache: "cnf-dnscache"
   dnsOverHttps:
    name: "doh-server"
   clientSideHttp2:
     concurrentStreamsPerConnection: 20
   clientSideHttp:
     maxHeaderSize: 32768
   clientSideSsl:
     enableRenegotiation: true
     keyCertPairs:
      - key: 'file://etc/ssl/tls-keys-certs/client.key'
        cert: 'file://etc/ssl/tls-keys-certs/client.crt'
   tcpSettings:
     clientSide: "tcp-profile-client"
   udpSettings:
     serverSide: "udp-profile-server"
   pool:
     minActiveMembers: 1
     members:
     - address: "10.244.99.110"
       port: 53
       priorityGroup: 1
   monitors:
     dns:
      enabled: true
      queryName: "webapp.net."
      queryType: "aaaa"
      recv: "2002::10:10:20:200"

  8. Install the F5BigDohApp CR:

    oc apply -f cnf-dohapp-cr.yaml

    In this example, the BIG-IP Controller logs indicate the F5BigDohApp CR was added/updated:

    I0208 12:00:00.12345   1 event.go:282] Event(v1.ObjectReference{Kind:"F5Doh",
F5Doh cnf-gateway/cnf-dohapp was added/updated

Traffic statistics

If you installed the CNF Controller with the Debug Sidecar enabled, connect to the sidecar to view the DNS statistics.

  1. Log in to the TMM Debug container:

    oc exec -it deploy/f5-tmm -c debug -n cnf-gateway -- bash

  2. View the IPS statstics:

    tmctl -d blade protocol_inspection_stats

    In this example, IPS disallowed MX records has matched 11 times, blacklisted domains 3:

    insp_id insp_name                       vs_name
------- ------------------------------- -------------------------------------
  10007 dns_disallowed_resource_records cnf-gateway-cnf-dohapp-virtual_server
  10009 dns_domains_blacklist           cnf-gateway-cnf-dohapp-virtual_server

prof_name                                     hit_count last_hit_time
--------------------------------------------- --------- -------------
cnf-gateway-dns-ips-profileprotocolinspection        11    1645748374
cnf-dateway-dns-ips-profileprotocolinspection         3    1645748620

  3. View the DNS caching statistics:

    tmctl -d blade dns_cache_resolver_stat -s name,queries,responses,msg.hits,msg.inserts

    In this example, 55 queries have been process, and 7 domain names have been added to the DNS cache.

    name                           queries responses msg.hits msg.inserts
------------------------------ ------- --------- -------- -----------
cnf-gateway-cnf-dnscache            55        48       48           7

  4. View the DNS resolution statistics:

    tmctl -d blade profile_dns_stat -s name,vs_name,queries

    In this example, 55 successful DNS queries have been processed:

    name                               vs_name                               queries
---------------------------------- ------------------------------------- -------
cnf-gateway-cnf-dohapp-profile_dns cnf-gateway-cnf-dohapp-virtual_server      55

Monitor status

When the F5BigDohApp has a monitor configured, the Service Proxy TMM Pod logs pool member status change messages similar to the following:

oc logs -f f5-tmm-7599d547fc-g2zqd -n cnf-gateway | grep 'Pool Member Status'

f5-tmm-7599d547fc-g2zqd tmm[34]: 01010057:3: Pool Member Status Change: pool member 2002::10:10:10:100:53 in cnf-gateway-cnf-dohapp-pool is up\n"
f5-tmm-7599d547fc-g2zqd tmm[34]: 01010057:3: Pool Member Status Change: pool member 2002::10:10:10:100:53 in cnf-gateway-cnfn-dohapp-pool is down\n"
f5-tmm-7599d547fc-g2zqd tmm[34]: 01010057:3: Pool Member Status Change: pool member 2002::10:10:10:100:53 in cnf-gateway-cnf-dohapp-pool is up\n"

Feedback

Provide feedback to improve this document by emailing cnfdocs@f5.com.

Supplemental