F5SPKEgressHTTP2

The F5SPKEgressHTTP2 Custom Resource (CR) configures the Service Proxy Traffic Management Microkernel (TMM) to proxy and forward 5G Service-Based Interface (SBI) messages over HTTP/2 to external destinations. It creates a Layer 7 HTTP/2 virtual server that acts as an egress gateway, enabling CNF pods inside the cluster to send HTTP/2 traffic outside the cluster. The F5SPKEgressHTTP2 CR supports noTLS/TLS/mTLS termination on both the client and server sides. For the full list of CRs, refer to the SPK CRs overview.

Parameters

The table below describes the F5SPKEgressHTTP2 CR parameters.

Parameter Description
sslFileWatchMode The type of file watching for any changes in the keys and certs used by client and server profiles in TMM: SSL_FILE_WATCH_MODE_NONE, SSL_FILE_WATCH_MODE_KUBERNETES_SECRET_STORE, or SSL_FILE_WATCH_MODE_FILES_IN_SHARED_VOLUME (default).

clientssl

Note: Refer to the Managing Certificates and Keys section prior to configuring the clientssl parameters.

Parameter Description
keyCertPairs.secretRef Specifies the Kubernetes Secret that contains the TLS certificates and key. For enhanced security, it is recommended to use the Secret key instead of providing the certificate and key in plain text. If both the Secret key and plain text certificate/key are specified, the Secret key will take precedence.
keyCertPairs.cert Specifies the certificate content, including any intermediate Certificate Authorities (CAs), if applicable. Supported formats include Embedded PEM or a file path. (Not required if secretRef is provided )
keyCertPairs.key Specifies the private key. Supported formats include Embedded PEM, Vault path, or file path. (Not required if secretRef is provided )
serverNames Specifies a list of hostnames or a domain used for Server Name Indication (SNI) matching to select the appropriate client SSL profile.
ciphers Specifies the OpenSSL-style cipher string: DEFAULT (default) or ALL.
enableClientAuthentication Enables or disables client certificate verification: true or false (default).
trustedCa Specifies a list of root CAs in PEM format used for client certificate verification.

For enhanced security, it is recommended to use the Secret key (keyCertPairs.secretRef) instead of providing the certificate and key in plain text. If both the Secret key and plain text certificate/key are specified, the Secret key will take precedence.

serverssl

Note: Refer to the Managing Certificates and Keys section prior to configuring the serverssl parameters.

Parameter Description
offload Specifies whether the TMM performs TLS offloading: true or false (default).
keyCertPairs.secretRef Specifies the Kubernetes Secret that contains the TLS certificates and key. For enhanced security, it is recommended to use the Secret key instead of providing the certificate and key in plain text. If both the Secret key and plain text certificate/key are specified, the Secret key will take precedence.
keyCertPairs.cert Specifies the certificate content, including any intermediate Certificate Authorities (CAs), if applicable. Supported formats include Embedded PEM or a file path.
keyCertPairs.key Specifies the private key. Supported formats include Embedded PEM, Vault path, or file path.
ciphers Specifies the OpenSSL-style cipher string: DEFAULT (default) or ALL.
enableServerAuthentication Enables or disables server certificate verification: true or false (default).
trustedCa Specifies a list of root CAs in PEM format used for server certificate verification.

For enhanced security, it is recommended to use the Secret key (keyCertPairs.secretRef) instead of providing the certificate and key in plain text. If both the Secret key and plain text certificate/key are specified, the Secret key will take precedence.

spec

Parameter Description
destinationSubnet Specifies the IPv4 subnet that listens for incoming network traffic and handles connection requests.
destinationPorts Specifies the list of ports that listen for incoming network traffic and process connection requests.
idleTimeout Specifies the time in seconds that a client connection can remain open without activity before closing. The default is 300 seconds.
v6destinationSubnet Specifies the IPv6 subnet that listens for incoming network traffic and processes connection requests.
vlans.vlanList Specifies a list of F5SPKVlan CRs to listen for egress traffic, using the CR's metadata.name. The list can also be disabled using disableListedVlans.
vlans.disableListedVlans Specifies whether to use all VLANs on the egress side except the listed ones true (default), or only the VLANs in the list false.
vlans.category Specifies an F5SPKVlan CR category to listen for egress traffic. The category can also be disabled using disableListedVlans.
ipfamilies Specifies the IP version capabilities of the application: IPv4 (default), IPv6, or IPv4andIPv6.
jsonMaxBytes The maximum number of bytes allowed for a JSON payload. The default is 65536.
jsonMaxEntries The maximum number of entries in the JSON payload. The default is 2048.
irule The iRule procedure called by the staticRoutes.customIruleProc parameter, which will be applied to matching connections.
snatPool If snatType is SRC_TRANS_SNATPOOL, this value specifies the name of the SNAT pool to use when forwarding requests that don’t match any static route.
snatType Specifies the type of SNAT to use when forwarding requests that don't match any static route: SRC_TRANS_NONE, SRC_TRANS_SNATPOOL, or SRC_TRANS_AUTOMAP (default).

spec.staticRoutes

Parameter Description
customIruleProc The name of the spec.irule to call when a matching condition occurs.
snatType Specifies the type of SNAT to use when forwarding requests that match this static route: SRC_TRANS_NONE, SRC_TRANS_SNATPOOL, or SRC_TRANS_AUTOMAP (default).
snatPool If snatType is SRC_TRANS_SNATPOOL, this value is the name of the SNAT pool to use when forwarding requests that match this static route.
conditions Specifies an array of up to four conditions that must be met for the static route to be selected and applied.
conditions.fieldName Specifies a string representing the name of a field in the message, such as an HTTP header or pseudo-header (for example, ':m'), or a JSON path (for example, ':JSON:some:path:in:json:payload').
conditions.comparisonOp Specifies a comparison operation to perform against the fieldName value to determine whether the condition is met: SR_COMPARE_NONE, SR_COMPARE_EQUALS (default), SR_COMPARE_NOT_EQUALS, SR_COMPARE_STARTS_WITH, SR_COMPARE_ENDS_WITH, SR_COMPARE_CONTAINS, SR_COMPARE_EXISTS, or SR_COMPARE_NOT_EXISTS.
conditions.values Specifies a list of constant values to compare against a fieldName value. When multiple values are provided, the condition will be met if any of them matches.
conditions.caseSensitive Enables or disables case-sensitivity evaluation of conditions: true (default) or false.

CR Examples

F5SPKEgressHTTP2 CR with secretRef

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKEgressHTTP2
metadata:
  name: "mtls"
  namespace: "spk-ingress"
clientssl:
  enableClientAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - secretRef:
      certField: tls.crt
      keyField: tls.key
      name: tls-keys-certs-secret
  ciphers: ALL
serverssl:
  offload: false
  enableServerAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - secretRef:
      certField: tls.crt
      keyField: tls.key
      name: tls-keys-certs-secret
  ciphers: ALL
spec:
  ipfamilies: "IPv4"
  snatPool: ""
  destinationPorts:
  - 21564
  idleTimeout: 300
  snatType: "SRC_TRANS_AUTOMAP"

F5SPKEgressHTTP2 CR with Plain Text (No TLS)

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKEgressHTTP2
metadata:
  name: "no-tls"
  namespace: "spk-ingress"
serverssl:
  offload: true
spec:
  ipfamilies: "IPv4"
  snatPool: ""
  destinationPorts:
  - 21543
  idleTimeout: 300
  snatType: "SRC_TRANS_AUTOMAP"

F5SPKEgressHTTP2 CR with Mutual TLS (mTLS)

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKEgressHTTP2
metadata:
  name: "mtls"
  namespace: "spk-ingress"
clientssl:
  enableClientAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/tls.key
    cert: file://etc/ssl/tls-keys-certs/tls.crt
  ciphers: ALL
serverssl:
  offload: false
  enableServerAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/tls.key
    cert: file://etc/ssl/tls-keys-certs/tls.crt
  ciphers: ALL
spec:
  ipfamilies: "IPv4"
  snatPool: ""
  destinationPorts:
  - 21564
  idleTimeout: 300
  snatType: "SRC_TRANS_AUTOMAP"

F5SPKEgressHTTP2 CR with mTLS Offload

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKEgressHTTP2
metadata:
  name: "mtls-offload"
  namespace: "spk-ingress"
clientssl:
  enableClientAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/tls.key
    cert: file://etc/ssl/tls-keys-certs/tls.crt
  ciphers: ALL
serverssl:
  offload: true
  enableServerAuthentication: false
  ciphers: ALL
spec:
  ipfamilies: "IPv4"
  snatPool: ""
  destinationPorts:
  - 21566
  idleTimeout: 300
  snatType: "SRC_TRANS_AUTOMAP"

Setting Up SPK Controller to Monitor Application Pods

The SPK Controller and Service Proxy TMM Pods are installed in a separate project from the application pods (TCP or HTTP/2 pods). When installing the SPK Controller, ensure that the namespaces of the application pods are included in the controller.watchNamespace parameter in the Helm values file, allowing the SPK Controller to monitor and manage these application pods.

Note: Ensure that all application namespaces are created before installing SPK.

controller:
  watchNamespace: 
    - "spk-app-1"
    - "spk-app-2"
    - "spk-app-3"
    - "spk-app-4"
    - "spk-app-5"

Managing Certificates and Keys

The noTLS/TLS/mTLS certificates and keys referenced by the F5SPKEgressHTTP2 CR must be correctly encoded and installed in the cluster. Consider the following key points:

  • When installing the SPK Controller you must set tmm.tlsStore.enabled paramter to true.

  • The noTLS/TLS/mTLS certificates and keys must be Base64 encoded and stored in a Secret named tls-keys-certs-secret.

  • TMM mounts the Secret named tls-keys-certs-secret to the file path file://etc/ssl/tls-keys-certs/.

Important: The tls-keys-certs-secret Secret must be created before the SPK Controller is installed, otherwise the mount will fail and cause the TMM to enter a restart loop.

The certificates can be created manually or using certificate manager.

Creating certificates manually

Follow these steps to generate a new noTLS/TLS/mTLS certificate and key, encode them in Base64, and create the tls-keys-certs-secret Secret to store them in the cluster. F5 recommends using noTLS/TLS/mTLS certificates signed by a trusted certificate authority (CA) for production application traffic.

Note: Follow steps 4 - 6 if you already have an existing noTLS/TLS/mTLS certificate and key pair.

  1. Generate the CA signing certificate and key.

    openssl genrsa -out ca.key 4096

    openssl req -x509 -new -nodes -key ca.key -sha256 -days 365 -out ca.crt \
-subj "/C=US/ST=WA/L=Seattle/O=F5/OU=Dev/CN=ca"

  2. Generate the clientssl profile noTLS/TLS/mTLS certificate signing request (CSR).

    openssl genrsa -out client.key 4096

    openssl req -new -key client.key -out client.csr \
-subj "/C=US/ST=WA/L=Seattle/O=F5/OU=PD/CN=client.com"

  3. Sign the clientssl profile CSR with the CA.

    openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key \
-set_serial 101 -outform PEM -out client.crt -extensions req_ext -days 365 -sha256

  4. Base64 encode the SSL/TLS certificate and key.

    openssl base64 -A -in client.crt -out client-encode.crt
openssl base64 -A -in client.key -out client-encode.key

  5. Create the tls-keys-certs-secret Secret that stores the noTLS/TLS/mTLS certificate and key.

    echo "apiVersion: v1" > tls-keys-certs-secret.yaml
echo "kind: Secret" >> tls-keys-certs-secret.yaml
echo "metadata:" >> tls-keys-certs-secret.yaml
echo " name: tls-keys-certs-secret" >> tls-keys-certs-secret.yaml
echo "data:" >> tls-keys-certs-secret.yaml
echo -n " client.crt: " >> tls-keys-certs-secret.yaml
cat client-encode.crt >> tls-keys-certs-secret.yaml
echo " " >> tls-keys-certs-secret.yaml
echo -n " client.key: " >> tls-keys-certs-secret.yaml
cat client-encode.key >> tls-keys-certs-secret.yaml

  6. Install the Secret into the SPK Controller Project.

    oc apply -f tls-keys-certs-secret.yaml -n spk-ingress

Creating certificates using certificate manager

Refer the following documentation for creating certificates using certificate manager.

https://clouddocs.f5.com/service-proxy/latest/spk-cert-manager.html

Requirements

Ensure you have:

  • Installed a K8S Service object and application.

  • Installed the SPK Controller.

  • A Linux based workstation.

Deployment

Follow these steps to deploy the F5SPKEgressHTTP2 CR.

  1. Copy the F5SPKEgressHTTP2 example CR below to a YAML file, and deploy it to the same Project as the SPK Controller.

    In this example, the CR installs to the spk-ingress Project.

    apiVersion: "k8s.f5net.com/v1"
kind: F5SPKEgressHTTP2
metadata:
  name: "mtls"
  namespace: "spk-ingress"
clientssl:
  enableClientAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/tls.key
    cert: file://etc/ssl/tls-keys-certs/tls.crt
  ciphers: ALL
serverssl:
  offload: false
  enableServerAuthentication: true
  trustedCa: file://etc/ssl/tls-keys-certs/ca.crt
  keyCertPairs:
  - key: file://etc/ssl/tls-keys-certs/tls.key
    cert: file://etc/ssl/tls-keys-certs/tls.crt
  ciphers: ALL
spec:
  ipfamilies: "IPv4"
  snatPool: ""
  destinationPorts:
  - 21564
  idleTimeout: 300
  snatType: "SRC_TRANS_AUTOMAP"

  2. Apply the F5SPKEgressHTTP2 CR that you have created.

    oc apply -f spk-egress-http2.yaml

Connection Statistics

If you installed the SPK Controller with the Debug Sidecar enabled, connect to the sidecar to view virtual server connectivity statistics.

Follow these steps to deploy the F5SPKEgressHTTP2 CR.

  1. Log in to the Service Proxy Debug container:

    oc exec -it deploy/f5-tmm -c debug -n spk-ingress -- bash

  2. View the virtual server connection statistics:

    tmctl -d blade virtual_server_stat -s name,serviceside.tot_conns

    For example:

    name                                    serverside.tot_conns
--------------------------------------- --------------------
egress-http2-mtls-vs                                     31

Feedback

Provide feedback to improve this document by emailing spkdocs@f5.com.

Supplemental