Upgrade SPK from v2.0.0 to v2.0.1

If your deployment is on a specific version and you want to upgrade, check the Release Notes to learn about the new features and fixes before upgrading. To successfully upgrade SPK from v2.0.0 to v2.0.1, follow the instructions provided in the following sections in the specified sequence.

  1. Upgrade the CRD Conversion pod and apply CRDs

  2. Upgrade the Cert Manager

  3. Upgrade the RabbitMQ

  4. Upgrade the CWC

  5. Upgrade the Toda-fluentd

  6. Upgrade the dSSM

  7. Upgrade the F5Ingress

  8. Upgrade the Coremond

Prereqisites:

Upgrade the CRD Conversion pod and apply CRDs

  1. Verify the crd_conv_overrides_2.0.1.yaml contents.

    cat crd_conv_overrides_2.0.1.yaml
    

    Sample Output:

    crdconversion:
      image:
        repository: repo.f5.com/images
     
      rabbitmqNamespace: spk-utilities
     
      fluentbit_sidecar:
        image:
          repository: repo.f5.com/images
        fluentd:
          host: f5-toda-fluentd.spk-utilities.svc.cluster.local.
    
  2. Upgrade the the CRD Conversion pod.

    In this example, the new version of f5-crdconversion helm chart is 0.16.15-0.0.15.

     helm upgrade f5-crd-conversion tar/f5-crdconversion-0.16.15-0.0.15.tgz -f crd_conv_overrides_2.0.1.yaml -n  spk-utilities         
    

    Sample Output:

    Release "f5-crd-conversion" has been upgraded. Happy Helming!
    NAME: f5-crd-conversion
    LAST DEPLOYED: Wed Jun  4 11:23:17 2025
    NAMESPACE: f5-certificate-orchestrator
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    
  3. Upgrade the common CRDs.

    helm upgrade f5-spk-crds-common tar/f5-spk-crds-common-8.5.2-0.1.12.tgz -f crds_overrides.yaml --dry-run
    

    Sample Output:

    Release "f5-spk-crds-common" has been upgraded. Happy Helming!
    NAME: f5-spk-crds-common
    LAST DEPLOYED: Wed Jun  4 07:45:39 2025
    NAMESPACE: default
    STATUS: pending-upgrade
    REVISION: 2
    TEST SUITE: None
    
  4. Upgrade the SPK CRDs.

    helm upgrade tar/f5-spk-crds-service-proxy-8.5.2-0.1.12.tgz -f crds_overrides.yaml
    

    Sample Output:

    Release "f5-spk-crds-service-proxy" has been upgraded. Happy Helming!
    NAME: f5-spk-crds-service-proxy
    LAST DEPLOYED: Wed Jun  4 07:44:26 2025
    NAMESPACE: default
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    

Upgrade the Cert Manager

  1. Verify the cert-manager.yaml contents.

    cat cert-manager.yaml
    

    Sample Output:

    image:
      repository: repo.f5.com/images
     
    webhook:
      image:
        repository: repo.f5.com/images
     
    cainjector:
      image:
        repository: repo.f5.com/images
     
    startupapicheck:
      image:
        repository: repo.f5.com/images
     
    init_container:
      image:
        name: init-certmgr
        repository: repo.f5.com/images
    logging_sidecar:
      # Enable/Disable logging sidecar
      enabled: false
      name: logging-sidecar
      image:
        name: f5-fluentbit
        repository: repo.f5.com/images
     
      fluentbit:
        input:
          pipes:
            bufSize: 8096
        tls:
          enabled: true
      fluentd:
        host: f5-toda-fluentd.spk-utilities.svc.cluster.local.
    
  2. Upgrade the Cert Manager.

    helm upgrade f5-certificate-manager tar/f5-cert-manager-0.23.28-0.0.12.tgz -n spk-utilities   -f cert-manager.yaml
    

    Sample Output:

    Release "f5-certificate-manager" has been upgraded. Happy Helming!
    NAME: f5-certificate-manager
    LAST DEPLOYED: Wed Jun  4 08:42:49 2025
    NAMESPACE: f5-certificate-orchestrator
    STATUS: pending-upgrade
    REVISION: 2
    TEST SUITE: None
    HOOKS:
    
  3. See the installed releases in spk-utilities namespace to check for newly installed Cert Manager.

    helm list -n spk-utilities
    

Upgrade the RabbitMQ

  1. Verify the rmq_overrides.yaml contents.

    cat rmq_overrides.yaml
    

    Sample Output:

    image:
      repository: repo.f5.com/images
     
    fluentbit_sidecar:
      image:
        repository: repo.f5.com/images
    
  2. Upgrade the RabbitMQ.

    helm upgrade rabbitmq tar/rabbitmq-0.5.10-10.0.17.tgz -n spk-utilities -f rmq-values.yaml 
    

    Sample Output:

    Release "rabbitmq" has been upgraded. Happy Helming!
    NAME: rabbitmq
    LAST DEPLOYED: Tue Apr 29 07:15:41 2025
    NAMESPACE: spk-utilities
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    NOTES:
    The RabbitMQ has been installed.
    
  3. View the list of pods associated with spk-utilities namespace/project.

    oc get pods -n spk-utilities
    

Upgrade the CWC

  1. Verify the cwc_overrides.yaml contents.

    cat cwc_overrides.yaml
    

    Sample Output:

    cwc:
      image:
        repository: repo.f5.com/images
    orch:
      image:
        repository: repo.f5.com/images
     
    fluentbit_sidecar:
      enabled: true
      image:
          repository: repo.f5.com/images
    
  2. Upgrade the CWC.

    **Note: Make sure to replace cpclConfig.jwt value in the command with a valid JWT token.

    helm upgrade cwc tar/cwc-0.41.34-10.0.22.tgz -n spk-utilities -f cwc_overrides.yaml --set rabbitmqNamespace=spk-utilities --set cpclConfig.jwt="<JWT Token>"
    

    Sample Output:

    Release "cwc" has been upgraded. Happy Helming!
    NAME: cwc
    LAST DEPLOYED: Tue Apr 29 09:37:43 2025
    NAMESPACE: spk-utilities
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    NOTES:
    The Cluster Wide Controller has been installed.
    ====================================================
    Admin Token Feature: DISABLED
    No authentication token is required for REST API calls.
    ====================================================
    
  3. View the list of pods associated with spk-utilities namespace/project.

    oc get pods -n spk-utilities
    
  4. Check the license status after the CWC upgrade, see License status. If the license has expired, renew it before proceeding with the F5Ingress upgrade.

Upgrade the Toda-fluentd

  1. Verify the toda.yaml contents.

    cat toda.yaml
    

    Sample Output:

    image:
      repository: repo.f5.com/images
      pullPolicy: Always
     
    dssm_logs:
      enabled: true
      stdout: true
     
    f5ingress_logs:
      enabled: true
      stdout: true
     
    dssm_sentinel_logs:
      enabled: true
      stdout: true
     
    persistence:
      enabled: true
     
    tls:
      enabled: true
    
  2. Upgrade the Toda-fluentd.

    helm upgrade f5-toda-fluentd tar/f5-toda-fluentd-1.31.12-10.0.13.tgz -f toda.yaml -n spk-utilities
    

    Sample Output:

    Release "f5-toda-fluentd" has been upgraded. Happy Helming!
    NAME: f5-toda-fluentd
    LAST DEPLOYED: Tue Apr 29 10:53:09 2025
    NAMESPACE: spk-utilities
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    NOTES:
    Log aggregator - FluentD is deployed, which get logs from fluentbit sidecars.
    FluentD outputs:
        'stdout' is "true"
        'persistent volume' is "true"
    Persistent volume claim created with:
        accessModes: "ReadWriteOnce"
        storage: "3Gi"
        
    FluentD hostname: f5-toda-fluentd.spk-utilities.svc.cluster.local.
    FluentD port: "54321"
    Use this info to connect to it:   
    --set f5-toda-logging.fluentd.host="f5-toda-fluentd.spk-utilities.svc.cluster.local."
    --set f5-toda-logging.fluentd.port=54321
    
    FluentD service IP family:        
        serviceIpFamily: .Values.serviceIpFamily
    
  3. See the installed releases in spk-utilities namespace to check for newly installed Cert Manager.

    helm list -n spk-utilities
    
  4. View the list of pods associated with spk-utilities namespace/project.

    oc get pods -n spk-utilities
    

Upgrade the dSSM

To successfully upgrade dSSM, follow the instructions in Upgrading dSSM.

Upgrade the F5Ingress

With the implementation of readinessGates (config and routing gates), we have ensured TMM is available to process the network traffic with minimal traffic loss. For more information, see TMM Rolling Update.

Important: Based on the maxUnavailable, maxSurge, and TMM replicas configuration, ensure that you have:

  • Enough resources available.

  • Additional SelfIPs and translationIPs (SNAT and CGNAT).

  1. Verify the overrides_2.0_values.yaml contents.

    Note: The following parameters are configured to ensure that, after an upgrade, the TMM pod is immediately available to receive network traffic without any traffic loss:

    • tmm.bfdToOVN.enabled is set to True

    • tmm.dynamicRouting.bfd is configured.

    cat overrides_2.0_values.yaml
    

    Sample Output:

    # This file contains overrides for ocp f5ingress chart in cnab
    f5-toda-logging:
      enabled: true
      fluentd:
        host: f5-toda-fluentd.spk-utilities.svc.cluster.local.
        port: 54321
      fluentbit:
        logLevel: debug
        tls:
          enabled: true
      sidecar:
        image:
          repository: repo.f5.com/images
      tmstats:
        enabled: true
        config:
          image:
            repository: repo.f5.com/images
     
    tmm:
      image:
        repository: repo.f5.com/images
      tlsStore:
        enabled: true
     
      logLevel: INFO
     
      k8sprobes:
        enabled: true
     
      grpc:
        enabled: true
     
      replicaCount: 2
     
      strategy:
        type: RollingUpdate
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
     
      nodeSelector:
          tmmnode: enabled
      bfdToOVN:
        enabled: true
     
      sessiondb:
        useExternalStorage: "true"
     
      dynamicRouting:
        enabled: true
        tmmRouting:
          config:
            bgp:
              asn: 64522
              bgpSecret: bgp-secret
              gracefulRestartTime: 120
              neighbors:
              - ip : 10.21.1.252
                asn: 64521
                acceptsIPv4: true
                fallover: true
              - ip :  fc21:1::253
                asn: 64521
                acceptsIPv6: true
                fallover: true
            bfd:
              interface: external
              interval: 100
              minrx: 100
              multiplier: 3
          image:
            repository: repo.f5.com/images
        tmrouted:
          image:
            repository: repo.f5.com/images
     
      cniNetworks: "spk-ingress/spk-ingress-internal-sriov,spk-ingress/spk-ingress-external1-sriov"
     
     
      customEnvVars:
      - name: SESSIONDB_EXTERNAL_SERVICE
        value: "f5-dssm-sentinel.spk-utilities"
      - name: SESSIONDB_DISCOVERY_SENTINEL
        value: "true"
      - name: OPENSHIFT_VFIO_RESOURCE_1
        value: "sriovEns21f0Mlx6NetdevPolicy"
      - name: OPENSHIFT_VFIO_RESOURCE_2
        value: "sriovEns21f1Mlx6NetdevPolicy"
      - name: SSL_SERVERSIDE_STORE
        value: "/tls/tmm/mds/clt"
      - name: SSL_TRUSTED_CA_STORE
        value: "/tls/tmm/mds/clt"
      - name: TMM_DEFAULT_MTU
        value: "9000"
      - name: CONFIG_VIEWER_ENABLE
        value: "TRUE"
     
      # vxlan
      vxlan:
       enabled: false
     
     
      icni2:
        enabled: true
     
      network:
        vfio:
          enabled: false
        attachment:
          definitionName: spk-ingress/internal-sriov
     
    f5-stats_collector:
      enabled: true
      image:
        repository: repo.f5.com/images
      stats_collector:
        image:
          repository: repo.f5.com/images
     
    controller:
      annotationDelay: 120
      cwcNamespace: spk-utilities
      watchNamespace: "dav21-appns-1,dav21-appns-2"
      enableCustomResources: false
      image:
        repository: repo.f5.com/images
      vlan_grpc:
        enabled: true
      fluentbit_sidecar:
        enabled: true
        fluentd:
          host: f5-toda-fluentd.spk-utilities.svc.cluster.local
        fluentbit:
          tls:
            enabled: true
        image:
            repository: repo.f5.com/images
      f5_lic_helper:
        enabled: true
        name: f5-lic-helper
        rabbitmqNamespace: spk-utilities
        image:
            repository: repo.f5.com/images
      tmm_pod_manager:
        enabled: true
        image:
          repository: repo.f5.com/images
     
    debug:
      image:
        repository: repo.f5.com/images
      rabbitmqNamespace: spk-utilities
     
    afm:
      name: f5-afm
      enabled: false
    
  2. (Optional) If you want to use the Multiple External Gateway (MEG) with SPK v2.0.1, apply the AdminPolicyBasedExternalRoute CR. For more information, see AdminPolicyBasedExternalRoute.

  3. (Optional) If you have f5-afm pod enabled in values.yaml, add privileges to the f5-afm service account.

    oc adm policy add-scc-to-user privileged -n spk-ingress -z f5-afm
    
  4. Upgrade the F5Ingress using the new f5ingress helm chart version mentioned in the SPK v2.0.1 tarball.

    Note: Before upgrading F5Ingress, ensure that readinessGates.enabled is set to true in values.yaml.

    helm upgrade f5ingress tar/f5ingress-<version>.tgz -f <values>.yaml -n namespace
    

    In this example, the Pods will be upgraded using the f5ingress-v0.761.1-0.0.238 Helm chart.

    helm upgrade f5ingress tar/f5ingress-v0.761.1-0.0.238.tgz -f overrides_2.0_values.yaml -n spk-ingress
    

    Sample Output:

    Release "f5ingress" has been upgraded. Happy Helming!
    NAME: f5ingress
    LAST DEPLOYED: Tue Apr 29 11:37:50 2025
    NAMESPACE: spk-ingress
    STATUS: deployed
    REVISION: 2
    TEST SUITE: None
    NOTES:
    The F5Ingress Controller has been installed.
    TMM debug sidecar is deployed. To access: kubectl exec -it deployment/f5-tmm -c debug -n spk-ingress -- bash
    
    Note: Need to use extra vlan IP and extra snat IP for the traffic to work seamlessly after upgrade. with maxSurge 1 and maxUnavailable 0.
    
  5. View the list of pods associated with spk-ingress namespace/project.

    oc get pods -n spk-ingress -o wide
    
  6. See the installed releases in spk-ingress namespace.

    In this example, the below command shows the installed releases in spk-ingress Project.

    helm list -n spk-ingress
    
  7. During F5Ingress update process, run the below command to check the status of readinessGates (ConfigurationDone and RoutingDone gates).

    In this example, the f5-tmm-7fb766f798-6bgbk is TMM pod.

    oc describe pod f5-tmm-7fb766f798-6bgbk
    

    Sample Output:

    Readiness Gates:
      Type                Status
      ConfigurationDone   True
      RoutingDone         True
    

Upgrade the Coremond

  1. Verify the coremond_overrides.yaml contents.

    cat coremond_overrides.yaml
    

    Sample Output:

    image:
      repository: devrepo.f5.com/images
    
    persistence:
      storageClass: managed-nfs-storage
    
    resources:
      limits:
        cpu: "1"
        memory: 200
      requests:
        cpu: "0.5"
        memory: 50
    
    fluentbit_sidecar:
      image:
        repository: devrepo.f5.com/images  
      resources:
        limits:
          cpu: "0.5"
          memory: "512Mi"
        requests:
          cpu: "0.2"
          memory: "256Mi"
    
  2. Upgrade the Coremond.

    helm upgrade f5-coremond tar/coremond-0.7.27-10.0.14.tgz -n spk-utilities -f coremond-values.yaml 
    

    Sample Output:

    Release "f5-coremond" has been upgraded. Happy Helming!
    NAME: f5-coremond
    LAST DEPLOYED: Wed Jun  4 09:03:45 2025
    NAMESPACE: f5-certificate-orchestrator
    STATUS: pending-upgrade
    REVISION: 2
    TEST SUITE: None
    HOOKS:
    MANIFEST:
    
  3. View the list of pods associated with spk-utilities namespace/project.

    oc get pods -n spk-utilities