TMM Rolling Update

The TMM rolling update feature enables the smooth execution of F5 Ingress deployment updates, to ensure that the TMM pod is readily available to receive network traffic. This is achieved using readinessGates (config and routing gates) which ensures incremental replacement of existing TMM pods with new ones. Kubernetes waits for the new TMM pods before removing the old TMM pods. For more information about readinessGates, see TMM pod readiness.

The number of TMM pods upgrading at a time is dependent on maxUnavailable and maxSurge parameters defined in Kubernetes and the number of TMM replicas. Users can customize the default maxUnavailable and maxSurge settings in the values file. Refer to Step 8 below for an example override values file. For more information on maxUnavailable and maxSurge parameters, see Rolling Update Deployment.

Important: Based on the maxUnavailable, maxSurge, and TMM replicas configuration, ensure that you have:

  • Enough resources available.

  • Additional SelfIPs and translationIPs (SNAT and CGNAT).

To upgrade the F5Ingress, follow the instructions below:

  1. Install the CRD Conversion pod.

    In this example, the new version of f5-crdconversion helm chart is 0.16.15-0.0.10.

    helm install crd-conv tar/f5-crdconversion-0.16.15-0.0.10.tgz -f crd-conv-values.yaml -n spk-ingress

    Sample Output:

    NAME: f5-crd-conversion
LAST DEPLOYED: Sat Apr  5 10:19:26 2025
NAMESPACE: spk-ingress
STATUS: deployed
REVISION: 1
TEST SUITE: None

  2. Run cat crd-conv-values.yaml to verify the crd-conv-values.yaml contents.

    Sample Output:

        crdconversion:
    image:
        repository: repo.f5.com/images

    rabbitmqNamespace: spk-ingress

    fluentbit_sidecar:
        image:
        repository: repo.f5.com/images

  3. Create a template for common CRDs from the CRD bundle.

    helm template tar/f5-spk-crds-common-8.5.2-0.1.4.tgz -f crd-values.yaml  > crd_commons-spk-utilities_conversionns.yaml

  4. Run the oc apply command to apply the Common CRDs template.

    oc apply -f crd_commons-spk-utilities_conversionns.yaml

    Sample Output:

    customresourcedefinition.apiextensions.k8s.io/f5-big-cne-addresslists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-cne-portlists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-context-globals.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-context-secures.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-ddos-globals.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-fw-policies.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-fw-rulelists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-log-hslpubs.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-log-profiles.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-bnkgateways.k8s.f5net.com unchanged
customresourcedefinition.apiextensions.k8s.io/l4routes.gateway.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-egresses.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-egressdiameters.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-egresshttp2s.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressdiameters.k8s.f5net.com configured
apiVersion: "k8s.f5net.com/v1"
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressegressudps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressgtps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresshttp2s.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressngaps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresssips.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresstcps.ingresstcp.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressudps.ingressudp.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-pools.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-servicetypelbippools.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-statefulsets.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-traffic-distributions.k8s.f5net.com created

  5. Create a template for SPK CRDs from the CRD bundle.

    helm template tar/f5-spk-crds-service-proxy-8.5.2-0.1.4.tgz -f crd-values.yaml  > crd_service-proxy-spk-utilities.yaml

  6. Run the oc apply command to apply the SPK CRDs template.

    oc apply -f crd_service-proxy-spk-utilities.yaml

    Sample Output:

    customresourcedefinition.apiextensions.k8s.io/f5-big-cne-addresslists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-cne-portlists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-context-globals.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-context-secures.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-ddos-globals.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-fw-policies.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-fw-rulelists.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-log-hslpubs.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-big-log-profiles.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-bnkgateways.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/l4routes.gateway.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-egresses.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-egressdiameters.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-egresshttp2s.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressdiameters.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressegressudps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressgtps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresshttp2s.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressngaps.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresssips.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingresstcps.ingresstcp.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-ingressudps.ingressudp.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-pools.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-servicetypelbippools.k8s.f5net.com configured
customresourcedefinition.apiextensions.k8s.io/f5-spk-statefulsets.k8s.f5net.com created
customresourcedefinition.apiextensions.k8s.io/f5-spk-traffic-distributions.k8s.f5net.com created

  7. Navigate to the directory that contains the latest SPK Software, then list the f5ingress Helm chart.

    cd cnfinstall; ls -1 tar | grep f5ingress

    Sample Output:

    f5ingress-9.0.0.tgz

  8. Obtain the Helm release name for the current SPK installation:

    In this example, the Helm release is in the spk-ingress namespace.

    helm list -n spk-ingress

    In this sample output, the Helm release is using CHART version f5ingress-8.0.0.

    NAME        NAMESPACE     REVISION    STATUS     CHART
f5ingress   spk-ingress     1           deployed   f5ingress-8.0.0

  9. Verify the overrides_2.0_values.yaml contents.

    Note: The following parameters are configured to ensure that, after an upgrade, the TMM pod is immediately available to receive network traffic without any traffic loss:

    • tmm.bfdToOVN.enabled is set to True

    • tmm.dynamicRouting.bfd is configured.

    cat overrides_2.0_values.yaml

    Sample Output:

    # This file contains overrides for ocp f5ingress chart in cnab
f5-toda-logging:
  enabled: true
  fluentd:
    host: f5-toda-fluentd.spk-utilities.svc.cluster.local.
    port: 54321
  fluentbit:
    logLevel: debug
    tls:
      enabled: true
  sidecar:
    image:
      repository: repo.f5.com/images
  tmstats:
    enabled: true
    config:
      image:
        repository: repo.f5.com/images
 
tmm:
  image:
    repository: repo.f5.com/images
  tlsStore:
    enabled: true
 
  logLevel: INFO
 
  k8sprobes:
    enabled: true
 
  grpc:
    enabled: true
 
  replicaCount: 2
 
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
 
  nodeSelector:
      tmmnode: enabled
  bfdToOVN:
    enabled: true
 
  sessiondb:
    useExternalStorage: "true"
 
  dynamicRouting:
    enabled: true
    tmmRouting:
      config:
        bgp:
          asn: 64522
          bgpSecret: bgp-secret
          gracefulRestartTime: 120
          neighbors:
          - ip : 10.21.1.252
            asn: 64521
            acceptsIPv4: true
            fallover: true
          - ip :  fc21:1::253
            asn: 64521
            acceptsIPv6: true
            fallover: true
        bfd:
          interface: external
          interval: 100
          minrx: 100
          multiplier: 3
      image:
        repository: repo.f5.com/images
    tmrouted:
      image:
        repository: repo.f5.com/images
 
  cniNetworks: "spk-ingress/spk-ingress-internal-sriov,spk-ingress/spk-ingress-external1-sriov"
 
 
  customEnvVars:
  - name: SESSIONDB_EXTERNAL_SERVICE
    value: "f5-dssm-sentinel.spk-utilities"
  - name: SESSIONDB_DISCOVERY_SENTINEL
    value: "true"
  - name: OPENSHIFT_VFIO_RESOURCE_1
    value: "sriovEns21f0Mlx6NetdevPolicy"
  - name: OPENSHIFT_VFIO_RESOURCE_2
    value: "sriovEns21f1Mlx6NetdevPolicy"
  - name: SSL_SERVERSIDE_STORE
    value: "/tls/tmm/mds/clt"
  - name: SSL_TRUSTED_CA_STORE
    value: "/tls/tmm/mds/clt"
  - name: TMM_DEFAULT_MTU
    value: "9000"
  - name: CONFIG_VIEWER_ENABLE
    value: "TRUE"
 
  # vxlan
  vxlan:
   enabled: false
 
 
  icni2:
    enabled: true
 
  network:
    vfio:
      enabled: false
    attachment:
      definitionName: spk-ingress/internal-sriov
 
f5-stats_collector:
  enabled: true
  image:
    repository: repo.f5.com/images
  stats_collector:
    image:
      repository: repo.f5.com/images
 
controller:
  annotationDelay: 120
  cwcNamespace: spk-utilities
  watchNamespace: "dav21-appns-1,dav21-appns-2"
  enableCustomResources: false
  image:
    repository: repo.f5.com/images
  vlan_grpc:
    enabled: true
  fluentbit_sidecar:
    enabled: true
    fluentd:
      host: f5-toda-fluentd.spk-utilities.svc.cluster.local
    fluentbit:
      tls:
        enabled: true
    image:
        repository: repo.f5.com/images
  f5_lic_helper:
    enabled: true
    name: f5-lic-helper
    rabbitmqNamespace: spk-utilities
    image:
        repository: repo.f5.com/images
  tmm_pod_manager:
    enabled: true
    image:
      repository: repo.f5.com/images
 
debug:
  image:
    repository: repo.f5.com/images
  rabbitmqNamespace: spk-utilities
 
afm:
  name: f5-afm
  enabled: false

  10. (Optional) If you want to use the Multiple External Gateway (MEG) with SPK v2.0.0, apply the AdminPolicyBasedExternalRoute CR. For more information, see AdminPolicyBasedExternalRoute.

  11. (Optional) If you have f5-afm pod enabled in values.yaml, add privileges to the f5-afm service account.

    oc adm policy add-scc-to-user privileged -n spk-ingress -z f5-afm

  12. Upgrade the F5Ingress using the new f5ingress helm chart version mentioned in the SPK v2.0.0 tarball.

    helm upgrade f5ingress tar/f5ingress-<version>.tgz -f <values>.yaml -n namespace

    In this example, the Pods will be upgraded using the f5ingress-9.0.0.tgz Helm chart.

    helm upgrade f5ingress tar/f5ingress-v0.761.1-0.0.115.tgz -f values.yaml -n spk-ingress

  13. See the installed releases in spk-ingress namespace.

    In this example, the below command shows the installed releases in spk-ingressnamespace.

    helm list -n spk-ingress

    Sample Output:

    NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
f5ingress       spk-ingress     1               2025-02-20 06:55:05.040511141 +0000 UTC deployed        f5ingress-10.0.157      v10.0.157

  14. Verify that the Pods have a STATUS of Running.

    oc get pods -n spk-ingress -o wide

    Sample Output:

    NAME                                  READY   STATUS    RESTARTS      AGE   IP             NODE                          NOMINATED NODE   READINESS GATES
f5-afm-68d5fc75db-zrf4d               2/2     Running   0             13m   10.130.1.153   master-1.ocp21.pd.f5net.com   <none>           <none>
f5-tmm-7f988c98df-kbsg8              7/7     Running    10m   10.130.1.150   master-1.ocp21.pd.f5net.com   <none>          2/2
f5-tmm-7f988c98df-pfbkt               7/7     Running     8m   10.129.0.221   master-2.ocp21.pd.f5net.com   <none>           2/2
f5ingress-f5ingress-7d8b6cf86-qsn7b   5/5     Running   0             13m   10.128.1.140   master-3.ocp21.pd.f5net.com   <none>           <none>
otel-collector-74c76d445c-7hnbn       1/1     Running   0             13m   10.128.1.139   master-3.ocp21.pd.f5net.com   <none>           <none>