F5SPKIngressHTTP2

The F5SPKIngressHTTP2 CR configures the Traffic Management Microkernel (TMM) to proxy and load balance low-latency 5G Service Based Interface (SBI) messages using an HTTP/2 protocol virtual server, and a load balancing pool consisting of 5G Network Function endpoints. The F5SPKIngressHTTP2 CR fully supports SSL/TLS termination, bi-directional traffic flow, and connection persistence based on network packet headers, footers, and JSON contents. For the full list of CRs, refer to the BIG-IP Next for Kubernetes CRs overview.

CR parameters

The table below describes the CR parameters. Each heading below represents the top-level parameter element. For example, to set the Kubernetes services.name list, use spec.services.name.

Parameter	Description
sslFileWatchMode	The type of file watching for any changes in the keys and certs used by client and server profiles in TMM: SSL_FILE_WATCH_MODE_NONE, SSL_FILE_WATCH_MODE_KUBERNETES_SECRET_STORE, or SSL_FILE_WATCH_MODE_FILES_IN_SHARED_VOLUME (default).

clientssl

Note: Refer to the Managing SSL/TLS certs and keys section prior to configuring the clientssl parameters.

Parameter	Description
keyCertPairs.cert	References SSL/TLS certificates and intermediate CA certificates used to terminate secure ingress connections. Certificate names must be appended to the path file://etc/ssl/tls-keys-certs/<name>.crt.
keyCertPairs.key	References SSL/TLS private keys. Key names must be appended to the path file://etc/ssl/tls-keys-certs/<name>.key.
serverNames	Specifies a list of hostnames, or a fully qualified domain name used to select the profile by server name indication (SNI) matching.
ciphers	Specifies the OpenSSL style cipher string: DEFAULT (default) or ALL.
enableClientAuthentication	Enables client certificate verification: true or false (default).
trustedCa	Specifies list of Root CAs in PEM format used for client certificate verification.

serverssl

Note: Refer to the Managing SSL/TLS certs and keys section prior to configuring the serverssl parameters.

Parameter	Description
offload	Specifies if the TMM does TLS offloading: true or false (default).
keyCertPairs.cert	References SSL/TLS certificates and intermediate CA certificates used when TMM behaves as an SSL client. Certificate names must be appended to the path file://etc/ssl/tls-keys-certs/<name>.crt.
keyCertPairs.key	References the SSL/TLS private keys matching the keyCertPairs.key certificates. Key names must be appended to the path file://etc/ssl/tls-keys-certs/<name>.key.
ciphers	Specifies the OpenSSL style cipher string: DEFAULT (default) or ALL.
enableServerAuthentication	Enables server certificate verification: true or false (default).
trustedCa	Specifies the list of Root CAs used for server certificate verification.

spec

Parameter	Description
services.name	A list of Service object names for the internal applications (Pods). The Controller creates a load balancing pool using the discovered Service Endpoints.
services.port	The exposed port for the service.
destinationAddress	Creates an IPv4 virtual server address to receive ingress connections.
destinationPort	Creates a service port to receive ingress connections. When the Kubernetes service being load balanced has multiple ports, install one CR per service, or use port 0 for all ports.
v6destinationAddress	Creates an IPv6 virtual server address to receive ingress connections.
loadBalancingMethod	Specifies the load balancing method used to distribute traffic across pool members: ROUND_ROBIN distributes connections evenly across all pool members (default), and RATIO_LEAST_CONN_MEMBER distributes connections first to members with the least number of active connections.
ingressVlans.vlanList	Specifies a list of F5SPKVlan CRs to listen for ingress traffic, using the CR's metadata.name. The list can also be disabled using disableListedVlans.
ingressVlans.category	Specifies an F5SPKVlan CR category to listen for ingress traffic. The category can also be disabled using disableListedVlans.
ingressVlans.disableListedVlans	Disables the VLANs specified with the vlanList parameter: true (default) or false. Excluding one VLAN may simplify having to enable many VLANS.
egressVlans.vlanList	Specifies a list of F5SPKVlan CRs to listen for egress traffic, using the CR's metadata.name. The list can also be disabled using disableListedVlans.
egressVlans.category	Specifies an F5SPKVlan CR category to listen for egress traffic. The category can also be disabled using disableListedVlans.
egressVlans.disableListedVlans	Disables the VLANs specified with the vlanList parameter: true (default) or false. Excluding one VLAN may simplify having to enable many VLANS.
idleTimeout	Specifies the time in seconds that a client connection may remain open without activity before closing. The default is 300.
ipfamilies	Specifies the IP version capabilities of the application: IPv4 (default), IPv6, or IPv4andIPv6.
jsonMaxBytes	The maximum number of bytes for a JSON payload. The default is 65536.
jsonMaxEntries	The maximum number of entries in the JSON payload. The default is 2048.
irule	The iRule proc (procedure) called by the staticRoutes.customIruleProc parameter that will apply to matching connections.

spec.staticRoutes

The spec.staticRoutes parameter defines packet matching conditions that steer traffic to the configured service endpoints. The conditions are applied in the order defined.

Parameter	Description
conditions	Specifies an array of up to 4 conditions that must be met for the staticRoute to be selected and applied.
conditions.fieldName	Specifies a string representing the name of a field in the message, such as an HTTP header or pseudo header. For example, ':m', or JSON path. For example, ':JSON:some:path:in:json:payload'.
conditions.comparisonOp	Specifies a comparison operation to perform against the fieldName value to determine whether the condition is met: SR_COMPARE_NONE, SR_COMPARE_EQUALS (default), SR_COMPARE_NOT_EQUALS, SR_COMPARE_STARTS_WITH, SR_COMPARE_ENDS_WITH, SR_COMPARE_CONTAINS, SR_COMPARE_EXISTS, or SR_COMPARE_NOT_EXISTS.
conditions.values	Specifies a list of constant values to compare against a fieldName value. When multiple are provided, the condition will be met when any value is matched.
conditions.caseSensitive	Enables evaluating conditions using case-sensitivity: true (default) or false.
customIruleProc	The name of the spec.irule to call when the condition match occurs.
persistBidirectional	Specifies whether persistence should be bidirectional when a packet match occurs: true (default) or false.
persistField	Specifies a custom field that matching requests use as a persistence key. The field specifies the name of an HTTP header or pseudo-header such as :m (method), or :u (uri). Paths to values in the JSON payload may also be specified such as :JSON:key1:key2, with each :key in the path navigating one level deeper into the JSON object tree.
persistTimeout	Specifies the persistence timeout for matching conditions. This value overrides any global timeout values. The default value 0 indicates this static route applies the global timeout value.
service	Specifies the name of the Kubernetes service that matching requests will be routed to. The service is selected from the list defined in the CR's spec.services.name parameter.
snatType	Specifies the type of SNAT to use when forwarding requests that match this static route: SRC_TRANS_NONE, SRC_TRANS_SNATPOOL, or SRC_TRANS_AUTOMAP (default). Note: Currently the Ingress usecase with SRC_TRANS_NONE type is not supported for LA.
snatPool	If snatType is SRC_TRANS_SNATPOOL, this value is the name of the snatpool to use when forwarding requests that match this static route.

CR example

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKIngressHTTP2
metadata:
  name: http2-app
  namespace: spk-app
clientssl:
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/client.key
    cert: file://etc/ssl/tls-keys-certs/client.crt
  ciphers: "DEFAULT"
spec:
  destinationAddress: 10.20.2.206
  destinationPort: 80
  v6destinationAddress: 2002::10:20:2:206
  irule: |
    proc insert_http_header_a {} {
      log local0. "insert a new HTTP header"
      HTTP::header insert "x-custom-proc" "sbi-A"
      set acctVal [MESSAGE::field value ":JSON:account"]
      log local0. "#########  acctValB = $acctVal"
    }
    proc insert_http_header_b {} {
      log local0. "insert a new HTTP header"
      HTTP::header insert "x-custom-proc" "sbi-B"
      set acctVal [MESSAGE::field value ":JSON:account"]
      log local0. "#########  acctValB = $acctVal"
    }
  ipfamilies: IPv4andIPv6
  loadBalancingMethod: "ROUND_ROBIN"
  services:
    - name: web-a
      port: 443
  services:
    - name: web-b
      port: 443
  staticRoutes:
  - customIruleProc: "insert_http_header_a"
    persistBidirectional: false
    persistField: "${:JSON:account}"
    persistTimeout: 60
    service: "web-a"
    conditions:
    - fieldName: ":JSON:account"
      comparisonOp: "SR_COMPARE_EQUALS"
      values:
      - "account1"
      caseSensitive: true
  - customIruleProc: "insert_http_header_b"
    persistBidirectional: false
    persistField: "${:JSON:account}"
    persistTimeout: 60
    service: "web-b"
    conditions:
    - fieldName: ":JSON:account"
      comparisonOp: "SR_COMPARE_EQUALS"
      values:
      - "account2"
      caseSensitive: true

Dual-Stack environments

TMM’s load balancing pool is created by discovering the Kubernetes Service Endpoints in the Project. In IPv4/IPv6 dual-stack environments, to populate the load balancing pool with IPv6 members, set the Service PreferDualStack parameter to IPv6. For example:

kind: Service
metadata:
  name: nginx-web-app
  namespace: spk-app
  labels:
    app: http2-web-app
spec:
  ipFamilyPolicy: PreferDualStack
  ipFamilies:
  - IPv6
  - IPv4

Important: When enabling PreferDualStack, ensure TMM’s internal F5SPKVlan interface configuration includes both IPv4 and IPv6 addresses.

Ingress traffic

To enable ingress network traffic, TMM must be configured to advertise virtual server IP addresses to external networks using the Border Gateway Protocol (BGP) dynamic routing protocol. Alternatively, you can configure appropriate routes on upstream devices. To configure BGP, see ZebOS ConfigMaps

Managing certs and keys

Read this section carefully to ensure the SSL/TLS certificates and keys referenced by the F5SPKIngressHTTP2 CR are encoded and installed into the cluster properly. These bullet points are essential:

• When installing the BIG-IP Next for Kubernetes you must set tmm.tlsStore.enabled paramter to true. See SPKInstance CR Parameters.
• The SSL/TLS certificates and keys must be Base64 encoded, and stored in a Secret named tls-keys-certs-secret.
• TMM mounts the Secret named tls-keys-certs-secret to the file path file://etc/ssl/tls-keys-certs/.

Important: The tls-keys-certs-secret Secret must be created before the BIG-IP Next for Kubernetes is installed, otherwise the mount will fail and cause the TMM to enter a restart loop.

Use the steps below to generate a new SSL/TLS certficate and key, Base64 encode them, and then create the tls-keys-certs-secret Secret to store them in the cluster. F5 recommends using SSL/TLS certificates signed by a well-known certificate authority (CA) for production application traffic.

Note: Use steps 4 - 6 if you already have an existing SSL/TLS certificate and key pair.

1. Generate the CA signing certificate and key:

   openssl genrsa -out ca.key 4096
   

   openssl req -x509 -new -nodes -key ca.key -sha256 -days 365 -out ca.crt \
   -subj "/C=US/ST=WA/L=Seattle/O=F5/OU=Dev/CN=ca"
   

2. Generate the clientssl profile SSL/TLS certificate signing request (CSR):

   openssl genrsa -out client.key 4096
   

   openssl req -new -key client.key -out client.csr \
   -subj "/C=US/ST=WA/L=Seattle/O=F5/OU=PD/CN=client.com"
   

3. Sign the clientssl profile CSR with the CA:

   openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key \
   -set_serial 101 -outform PEM -out client.crt -extensions req_ext -days 365 -sha256
   

4. Base64 encode the SSL/TLS certificate and key:

   openssl base64 -A -in client.crt -out client-encode.crt
   openssl base64 -A -in client.key -out client-encode.key
   

5. Create the tls-keys-certs-secret Secret that stores the SSL/TLS certificate and key:

   echo "apiVersion: v1" > tls-keys-certs-secret.yaml
   echo "kind: Secret" >> tls-keys-certs-secret.yaml
   echo "metadata:" >> tls-keys-certs-secret.yaml
   echo " name: tls-keys-certs-secret" >> tls-keys-certs-secret.yaml
   echo "data:" >> tls-keys-certs-secret.yaml
   echo -n " client.crt: " >> tls-keys-certs-secret.yaml
   cat client-encode.crt >> tls-keys-certs-secret.yaml
   echo " " >> tls-keys-certs-secret.yaml
   echo -n " client.key: " >> tls-keys-certs-secret.yaml
   cat client-encode.key >> tls-keys-certs-secret.yaml
   

6. Install the Secret into the BIG-IP Next for Kubernetes Controller Project:

   kubectl apply -f tls-keys-certs-secret.yaml -n default
   

Requirements

Ensure you have:

• Installed a K8S Service object and application.
• A Linux based workstation.
• Installed the dSSM Database to support persistence records.

Installation

Use the following steps to obtain the application’s Service object information to configure and install the F5SPKIngressHTTP2 CR.

1. Switch to the application Project:

   kubectl project <project>
   

   In this example, the application is in the spk-app Project:

   kubectl project spk-app
   

2. Use the Service object NAME and PORT to configure the CR service.name and service.port parameters:

   kubectl get service 
   

   In this example, the Service object NAMEs are http2-web-a and http2-web-b and the PORT is 443:

   NAME          TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S) 
   http2-web-a   NodePort   10.99.99.91   <none>        443:443/TCP
   http2-web-b   NodePort   10.99.99.99   <none>        443:443/TCP
   

3. Copy the example F5SPKIngressHTTP2 CR into a YAML file:

   apiVersion: "k8s.f5net.com/v1"
   kind: F5SPKIngressHTTP2
   metadata:
     name: http2-app
     namespace: spk-app
   clientssl:
     keyCertPairs:
     - key: file://etc/ssl/tls-keys-certs/client.key
       cert: file://etc/ssl/tls-keys-certs/client.crt
     ciphers: "DEFAULT"
   spec:
     destinationAddress: 10.20.2.206
     destinationPort: 80
     v6destinationAddress: 2002::10:20:2:206
     irule: |
       proc insert_http_header_a {} {
         log local0. "insert a new HTTP header"
         HTTP::header insert "x-custom-proc" "sbi-A"
         set acctVal [MESSAGE::field value ":JSON:account"]
         log local0. "#########  acctValA = $acctVal"
       }
       proc insert_http_header_b {} {
         log local0. "insert a new HTTP header"
         HTTP::header insert "x-custom-proc" "sbi-B"
         set acctVal [MESSAGE::field value ":JSON:account"]
         log local0. "#########  acctValB = $acctVal"
       }
     ipfamilies: IPv4andIPv6
     loadBalancingMethod: "ROUND_ROBIN"
     services:
       - name: http-web-a
         port: 443
     services:
       - name: http-web-b
         port: 443
     staticRoutes:
     - customIruleProc: "insert_http_header_a"
       persistBidirectional: false
       persistField: "${:JSON:account}"
       persistTimeout: 60
       service: "web-a"
       conditions:
       - fieldName: ":JSON:account"
         comparisonOp: "SR_COMPARE_EQUALS"
         values:
         - "account1"
         caseSensitive: true
     - customIruleProc: "insert_http_header_b"
       persistBidirectional: false
       persistField: "${:JSON:account}"
       persistTimeout: 60
       service: "web-b"
       conditions:
       - fieldName: ":JSON:account"
         comparisonOp: "SR_COMPARE_EQUALS"
         values:
         - "account2"
         caseSensitive: true
   

4. Install the F5SPKIngressHTTP2 CR:

   kubectl apply -f spk-ingress-http2.yaml
   

5. Web clients should now be able to connect to the application through the TMM.

Connection statistics

Connect to the Debug Sidecar in the TMM to view virtual server and pool member connectivity statistics.

1. Log in to the TMM Debug container:

   kubectl exec -it f5-tmm-6cdbc6bb65-j2r7d -c debug -n default -- bash
   

2. View the virtual server connection statistics:

   tmctl -d blade virtual_server_stat -s name,clientside.tot_conns
   

   For example:

   name                                    serverside.tot_conns
   --------------------------------------- --------------------
   spk-apps-nginx-http2-crd-virtual-server                 31
   

3. View the load balancing pool connection statistics:

   tmctl -d blade pool_member_stat -s pool_name,serverside.tot_conns
   

   For example:

   web-apps-http2-crd-pool                        15
   web-apps-http2-crd-pool                        16
   

Supplemental

• Kubernetes Service