Upgrade CNFs from 2.2.1 to 2.3.0 using Helm¶

To upgrade CNFs from 2.2.1 to 2.3.0, follow the instructions provided in this document in the specified order.

Upgrade the CRD Conversion pod and upgrade CRDs
Upgrade the Cert Manager
Upgrade the Coremond
Upgrade the RabbitMQ
Upgrade the CWC
Upgrade the Toda-fluentd
Upgrade the Toda-obserer
Upgrade the dSSM
Upgrade the Stats Collector
Upgrade the F5Ingress

Prereqisites:

Download the CNF manifest file, helm charts, docker images, and other utilities from CNFs Artifacts through F5 Artifact Registry.
Obtained JWT from MyF5.

Upgrade the CRD Conversion pod and upgrade CRDs¶

Verify the contents of crd_conv_overrides_2.3.0.yaml.

cat crd_conv_overrides_2.3.0.yaml

Sample Output:

crdconversion:
  image:
    repository: repo.f5.com/images

  rabbitmqNamespace: cnf-telemetry

  fluentbit_sidecar:
    image:
      repository: repo.f5.com/images
  fluentd:
    host: f5-toda-fluentd.cnf-gateway.svc.cluster.local.

Upgrade the CRD Conversion pod.

In the following example, the new version of f5-crdconversion helm chart is 0.81.1-0.0.4.

 helm upgrade f5-crd-conversion tar/f5-crdconversion-0.81.1-0.0.4.tgz -f crd_conv_overrides_2.3.0.yaml -n cnf-gateway         

Sample Output:

Release "f5-crd-conversion" has been upgraded. Happy Helming!
NAME: f5-crd-conversion
LAST DEPLOYED: Thu April 26 11:23:17 2026
NAMESPACE: cnf-gateway
STATUS: deployed
REVISION: 2
TEST SUITE: None

Verify the crds_overrides.yaml contents.

cat crds_overrides.yaml

Sample Output:

conversion:
  namespace: cnf-gateway

Upgrade the CNFs CRDs.

helm upgrade tar/f5-cnf-crds-n6lan-14.59.1-0.0.70.tgz -f crds_overrides.yaml

Sample Output:

Release "f5-cnf-crds-n6lan" has been upgraded. Happy Helming!
NAME: f5-cnf-crds-n6lan
LAST DEPLOYED: Thu April 26 07:44:26 2026
NAMESPACE: default
STATUS: deployed
REVISION: 2
TEST SUITE: None

Upgrade the Cert Manager¶

Verify the contents of cert-manager.yaml file.

cat cert-manager.yaml

Sample Output:

image:
  repository: repo.f5.com/images

webhook:
  image:
    repository: repo.f5.com/images

cainjector:
  image:
    repository: repo.f5.com/images

startupapicheck:
  image:
    repository: repo.f5.com/images

init_container:
  image:
    name: init-certmgr
    repository: repo.f5.com/images
logging_sidecar:
  # Enable/Disable logging sidecar
  enabled: false
  name: logging-sidecar
  image:
    name: f5-fluentbit
    repository: repo.f5.com/images

  fluentbit:
    input:
    pipes:
      bufSize: 8096
    tls:
      enabled: true
  fluentd:
    host: f5-toda-fluentd.cnf-gateway.svc.cluster.local.

Upgrade the Cert Manager.

helm upgrade f5-certificate-manager tar/f5-cert-manager-0.26.3-0.0.4.tgz -n cnf-gateway -f cert-manager.yaml

Sample Output:

Release "f5-certificate-manager" has been upgraded. Happy Helming!
NAME: f5-certificate-manager
LAST DEPLOYED: Thu April 26 08:42:49 2026
NAMESPACE: cnf-gateway
STATUS: pending-upgrade
REVISION: 2
TEST SUITE: None
HOOKS:

See the installed releases in cnf-gateway namespace to check for newly upgraded Cert Manager.
```
helm list -n cnf-gateway
```

Upgrade the Coremond¶

Verify the coremond-values.yaml contents.

cat coremond-values.yaml

Sample Output:

image:
  repository: devrepo.f5.com/images

persistence:
  storageClass: managed-nfs-storage

resources:
  limits:
    cpu: "1"
    memory: 200
  requests:
    cpu: "0.5"
    memory: 50

fluentbit_sidecar:
  image:
    repository: devrepo.f5.com/images  
  resources:
    limits:
      cpu: "0.5"
      memory: "512Mi"
    requests:
      cpu: "0.2"
      memory: "256Mi"

Upgrade the Coremond.

helm upgrade f5-coremond tar/coremond-v0.16.2.tgz -n cnf-gateway -f coremond-values.yaml 

Sample Output:

Release "f5-coremond" has been upgraded. Happy Helming!
NAME: f5-coremond
LAST DEPLOYED: Thu April 26 09:03:45 2026
NAMESPACE: cnf-gateway
STATUS: pending-upgrade
REVISION: 2
TEST SUITE: None
HOOKS:
MANIFEST:

View the list of pods associated with cnf-gateway namespace/project.
```
kubectl get pods -n cnf-gateway
```

Upgrade the RabbitMQ¶

Verify the contents of rabbitmq-values.yaml file.

cat rabbitmq-values.yaml

Sample Output:

image:
  repository: repo.f5.com/images

fluentbit_sidecar:
  image:
    repository: repo.f5.com/images

Upgrade the RabbitMQ.

helm upgrade rabbitmq tar/rabbitmq-0.10.3-0.0.3.tgz -n cnf-gateway -f rabbitmq-values.yaml

Sample Output:

Release "rabbitmq" has been upgraded. Happy Helming!
NAME: rabbitmq
LAST DEPLOYED: Thu April 26 07:15:41 2026
NAMESPACE: cnf-telemetry
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The RabbitMQ has been installed.

View the list of pods associated with cnf-telemetry namespace/project.
```
kubectl get pods -n cnf-telemetry
```

Upgrade the CWC¶

Prerequisites¶

Upgrade the CRD bundle, Cert Manager, CRD Conversion, and RabbitMQ pods before upgrading the CWC.
Verify the License CRD exists:
```
kubectl get crd | grep license
```
Verify no License resources exist yet:
```
kubectl get licenses.k8s.f5net.com -A
```

Convert CPCL ConfigMaps to Secrets¶

Run the cwc-secrets.sh script to convert CPCL ConfigMaps to Secrets.
```
cwc-secrets.sh -n <cwc-namespace>
```
Sample Output:

Verify the new secrets were created.

kubectl get secret -n <cwc-namespace>

You should see the following secrets:

cpcl-config-secret                  Opaque                                9      15s
cpcl-key-secret                     Opaque                                1      17s

Upgrade the CWC¶

Verify the contents of cwc_overrides.yaml file.

cat cwc_overrides.yaml

Sample Output:

cwc:
  image:
    repository: repo.f5.com/images
orch:
  image:
    repository: repo.f5.com/images

fluentbit_sidecar:
  enabled: true
  image:
    repository: repo.f5.com/images

Ensure the cwc_overrides.yaml file does not include the cpclConfig: section, which contains the JWT. The JWT is no longer passed through Helm values; it will be provided through a License CR instead.

Upgrade the CWC.

helm upgrade cwc tar/cwc-0.66.7-0.0.7.tgz -f cwc_overrides.yaml -n <cwc-namespace>

Sample Output:

Release "cwc" has been upgraded. Happy Helming!
NAME: cwc
LAST DEPLOYED: Thu April 26 09:37:43 2026
NAMESPACE: cnf-telemetry
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The Cluster Wide Controller has been installed.
====================================================
Admin Token Feature: DISABLED
No authentication token is required for REST API calls.
====================================================

Create the License CR¶

Create a License CR YAML file (for example, cwc-cr.yaml) using the JWT from the previously extracted cwc-values.yaml.

cat cwc-cr.yaml

Sample Output:

apiVersion: k8s.f5net.com/v1
kind: License
metadata:
  name: cnf-license
spec:
  jwt: "<JWT Token>"
  operationMode: "disconnected"

Apply the License CR.

kubectl apply -f cwc-cr.yaml -n <cwc-namespace>

Sample Output:

license.k8s.f5net.com/cnf-license created

Verify the License status.

kubectl get licenses.k8s.f5net.com -n <cwc-namespace>

Sample Output:

NAME          STATE    MODE           ENTITLEMENT   ENVIRONMENT   EXPIRY                 DIGITALASSETID                         AGE
cnf-license   Active   disconnected   paid          test          2027-03-30T07:05:27Z   a0691d08-1f57-4d7c-b91b-3c05f6685267   4s

Verify the CWC pod logs confirm license activation.

kubectl logs <cwc-pod-name> -n <cwc-namespace>

You should see log entries similar to:

"m"="Creating CM20 Response"|"responsetype"="ResponseCM20LicenseVerified"|"entitlement"="paid"
"m"="License-Controller Status sync: License CR updated"|"state"="Active"|"description"="License verification complete"

View the list of pods associated with the CWC namespace.
```
kubectl get pods -n <cwc-namespace>
```
Check the license status after the CWC upgrade, see License status. If the license has expired, renew it before proceeding with the F5Ingress upgrade.

Upgrade the Toda-fluentd¶

Verify the contents of toda.yaml file.

cat toda.yaml

Sample Output:

image:
  repository: repo.f5.com/images
  pullPolicy: Always

dssm_logs:
  enabled: true
  stdout: true

f5ingress_logs:
  enabled: true
  stdout: true

dssm_sentinel_logs:
  enabled: true
  stdout: true

persistence:
  enabled: true

tls:
  enabled: true

Upgrade the Toda-fluentd.

helm upgrade f5-toda-fluentd tar/f5-toda-fluentd-2.5.0-0.0.4.tgz -f toda.yaml -n cnf-gateway

Sample Output:

Release "f5-toda-fluentd" has been upgraded. Happy Helming!
NAME: f5-toda-fluentd
LAST DEPLOYED: Thu April 26 10:53:09 2026
NAMESPACE: cnf-gateway
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
Log aggregator - FluentD is deployed, which get logs from fluentbit sidecars.
FluentD outputs:
    'stdout' is "true"
    'persistent volume' is "true"
Persistent volume claim created with:
    accessModes: "ReadWriteOnce"
    storage: "3Gi"
    
FluentD hostname: f5-toda-fluentd.cnf-gateway.svc.cluster.local.
FluentD port: "54321"
Use this info to connect to it:   
--set f5-toda-logging.fluentd.host="f5-toda-fluentd.cnf-gateway.svc.cluster.local."
--set f5-toda-logging.fluentd.port=54321

FluentD service IP family:        
    serviceIpFamily: .Values.serviceIpFamily

See the installed releases in cnf-gateway namespace to check for newly installed Toda-fluentd.
```
helm list -n cnf-gateway
```
View the list of pods associated with cnf-gateway namespace/project.
```
kubectl get pods -n cnf-gateway
```

Upgrade the Toda-observer¶

Verify the contents of toda-observer.yaml file.

cat toda-observer.yaml

Sample Output:

imagePullSecrets:
  - name: cnabimagepullsecret
targets_namespaces:
  - cnf-gateway
watchNamespaces:
  - cnf-gateway
tolerations: 
  observer:
    - key: "node.kubernetes.io/not-ready"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30       
    - key: "node.kubernetes.io/unreachable"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30
  operator:
    - key: "node.kubernetes.io/not-ready"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30       
    - key: "node.kubernetes.io/unreachable"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30
  receiver: 
    - key: "node.kubernetes.io/not-ready"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30       
    - key: "node.kubernetes.io/unreachable"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30
  
image:
  repository: repo.f5.com/images
  pullPolicy: Always

revisionHistoryLimit: 10

replicas: 1

resources:
  observer: 
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 1
      memory: 1Gi
  operator: 
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 1
      memory: 1Gi
  receiver:
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 1
      memory: 1Gi

persistence:
  enabled: true
  storageClassName: managed-nfs-storage
  size: 3Gi
platformType: other  

## Fluentbit sidecar values
fluentbit_sidecar:
  image:
    repository: repo.f5.com/images
    pullPolicy: Always    
  resources:
    limits:
      cpu: "0.5"
      memory: "512Mi"
    requests:
      cpu: "0.25"
      memory: "256Mi"
  fluentbit:
    tls:
      enabled: true
  fluentd:
    host: f5-toda-fluentd.cnf-gateway.svc.cluster.local
    port: 54321

Note: To upgrade the toda-observer, delete the existing StatefulSet due to the addition of volumeClaimTemplates to the toda-observer. Kubernetes does not allow updates to StatefulSet specifications for fields, such as volumeClaimTemplates. Follow the below steps for a successful upgrade:

Use helm chart versions less than 4.0.

Delete the existing StatefulSet using the command.
kubectl delete sts f5-observer-receiver -n <namespace>

Perform the upgrade with the Helm command as shown below.
helm upgrade --install observer

Upgrade the Toda-observer.

helm upgrade f5-toda-observer tar/f5-toda-observer-5.30.13-0.0.5.tgz -f toda-observer.yaml -n cnf-gateway

Sample Output:

Release "f5-toda-observer" has been upgraded. Happy Helming!
NAME: f5-toda-observer
LAST DEPLOYED: Thu April 26 10:29:44 2026
NAMESPACE: svs-alpha
STATUS: deployed
REVISION: 2
TEST SUITE: None

See the installed releases in cnf-gateway namespace to check for newly installed Toda-observer.
```
helm list -n cnf-gateway
```
View the list of pods associated with cnf-gateway namespace/project.
```
kubectl get pods -n cnf-gateway
```

Upgrade the dSSM¶

To successfully upgrade dSSM, follow the instructions in Upgrading dSSM.

Upgrade the Stats Collector¶

Verify the contents of stats_collector_overrides.yaml file.

cat stats_collector_overrides.yaml

Sample Output:

enabled: true

image:
  repository: s5-cicd-registry.pdsea.f5net.com
  pullPolicy: Always

tolerations:
  - key: "node.kubernetes.io/not-ready"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 30       
  - key: "node.kubernetes.io/unreachable"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 30

global:
  core:
    streaming: true
    coremonAddr: f5-coremond.coremond:8088
  imageCredentials:
    name: cnabimagepullsecret

Upgrade the stats collector.

helm upgrade f5-stats_collector f5-stats_collector-1.0.48.tgz -n productns -f  stats_collector_overrides.yaml

Sample Output:

Release "f5-stats_collector" has been upgraded. Happy Helming!
NAME: f5-stats_collector
LAST DEPLOYED: Thu April 26 10:29:44 2026
NAMESPACE: svs-alpha
STATUS: deployed
REVISION: 2
TEST SUITE: None

See the installed releases in cnf-gateway namespace to check for newly installed Toda-observer.
```
helm list -n cnf-gateway
```
View the list of pods associated with cnf-gateway namespace/project.
```
kubectl get pods -n cnf-gateway
```

Upgrade the F5Ingress¶

With the implementation of readinessGates (config and routing gates), we ensured TMM is available to process the network traffic with minimal traffic loss. For more information, see TMM Rolling Update.

Important: Based on the maxUnavailable, maxSurge, and TMM replicas configuration, ensure that you have:

Enough resources available.

Additional SelfIPs and translationIPs (SNAT and CGNAT).

Verify the overrides_2.3.0_values.yaml contents.

Note: The following parameters are configured to ensure that, during the upgrade, the TMM pod is immediately available to receive network traffic with a very less traffic loss:

tmm.bfdToOVN.enabled is set to True

tmm.dynamicRouting.bfd is configured.

cat overrides_2.3.0_values.yaml

Sample Output:

Note: When upgrading the f5ingress, ensure the replicaCount value in the V1 deployment should match with the replicaCount value V2 helm chart. Any discrepancies between V1 and V2 replicaCount values can cause an error and result in a failed upgrade of the TMM pods while other pods may upgrade successfully, leading to an incomplete deployment.

# This file contains overrides for ocp f5ingress chart in cnab
f5-tmm:
  wholeClusterMode:
    enabled: false
  enabled: true
  tmm:
    logLevel: Debug
    add_k8s_routes: true
    cniNetworks: 'cnf-ingress/cnf-ingress-internal-sriov,cnf-ingress/cnf-ingress-external1-sriov'
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent    
    k8sprobes:
      enabled: true  
    replicaCount: 2
    strategy:
      rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
    resources:
      limits:
        cpu: "2"
        memory: "2Gi"
        hugepages-2Mi: "4Gi"
      requests:
        cpu: "2"
        memory: "2Gi"
    startupProbe:
      timeoutSeconds: 120
    grpc:
      enabled: true
      calico_routes: false 
    sessiondb:
      useExternalStorage: "true"
      extRetryWaitSec: "0"       
    dynamicRouting:
      enabled: true
      exportTmroutedLogs: true
      tmmRouting:
        config:
          bgp:
            asn: 64522
            bgpSecret: bgp-secret
            gracefulRestartTime: 120
            neighbors:
            - ip : 10.17.1.244
              asn: 64521
              acceptsIPv4: true
              fallover: true
            - ip : fc17:1::241
              asn: 64521
              acceptsIPv6: true
              fallover: true
          bfd:
          interface: external
          interval: 100
          minrx: 100
          multiplier: 3
        image:
          repository: repo.f5.com/images
      tmrouted:
        image:
          repository: repo.f5.com/images  
    customEnvVars: 
        - name: SESSIONDB_EXTERNAL_SERVICE
          value: "f5-dssm-sentinel.rajuocp-alpha.svc.cluster.local"
        - name: SESSIONDB_DISCOVERY_SENTINEL
          value: "true"
        - name: SSL_SERVERSIDE_STORE
          value: "/tls/tmm/mds/clt"
        - name: SSL_TRUSTED_CA_STORE
          value: "/tls/tmm/mds/clt"
        - name: TMM_MAPRES_VERBOSITY
          value: "debug"
        - name: TMM_MAPRES_USE_VETH_NAME  # Set to false to resolve the issue (BZ 1407137)
          value: "FALSE"            
        - name: CONFIG_VIEWER_ENABLE
          value: "TRUE"       
        - name: TMM_MAPRES_ADDL_VETHS_ON_DP
          value: "TRUE"
        - name: OPENSHIFT_VFIO_RESOURCE_1
          value: "sriovEns21f0np0f0Mlx5NetdevPolicy"
        - name: OPENSHIFT_VFIO_RESOURCE_2
          value: "sriovEns21f1np1Mlx5NetdevPolicy"      
        - name: EXPORT_TMROUTED_LOGS
          value: "true"  
        - name: EXPORT_BLOBD_LOGS
          value: "true"  

  debug:
    enabled: true
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent
    resources:
      limits:
        cpu: "0.5"
        memory: "1Gi"
      requests:
        cpu: "0.5"
        memory: "1Gi"
    rabbitmqNamespace: rabbitmq
  blobd:
    enabled: true 
    image:
      repository: repo.f5.com/images 
      pullPolicy: IfNotPresent
    resources:
      limits:
        cpu: "1"
        memory: "4Gi"
      requests:
        cpu: "1"
        memory: "4Gi"
  f5-toda-logging:
    enabled: true   
    tmstats:
      enabled: false
      config:
        image:
          repository: repo.f5.com/images
          pullPolicy: IfNotPresent
  fluentbit:
    enabled: true
    logLevel: debug
    tls:
      enabled: true
    fluentd:
      host: f5-toda-fluentd.cnf-gateway.svc.cluster.local.
      port: 54321
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent
  observer:
    enabled: true
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent 

f5-stats_collector:
  enabled: true
  image:
    repository: repo.f5.com/images
    pullPolicy: IfNotPresent  
  tolerations:
  - key: "node.kubernetes.io/not-ready"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 30       
  - key: "node.kubernetes.io/unreachable"
    operator: "Exists"
    effect: "NoExecute"
    tolerationSeconds: 30

controller:
  cwcNamespace: cnf-gateway
  image:
    repository: repo.f5.com/images
    pullPolicy: IfNotPresent  
  replicaCount: 1
  enableCustomResources: true
  f5_lic_helper:
    enabled: true
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent  
    name: f5-lic-helper
    rabbitmqNamespace: rabbitmq
  vlan_grpc:
    enabled: true
  fluentbit_sidecar:
    enabled: true
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent 
    fluentd:
      host: f5-toda-fluentd.cnf-gateway.svc.cluster.local.
      port: 54321
    fluentbit:
      tls:
        enabled: true
  crdupdater:
    enabled: true
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent
  tmm_pod_manager:
    image:
      repository: repo.f5.com/images
      pullPolicy: IfNotPresent
  tolerations: # check in the previous file / Sharan / Retain
    - key: "node.kubernetes.io/not-ready"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30       
    - key: "node.kubernetes.io/unreachable"
      operator: "Exists"
      effect: "NoExecute"
      tolerationSeconds: 30

(Optional) If you have f5-afm pod enabled in values.yaml, add privileges to the f5-afm service account.
```
kubectl adm policy add-scc-to-user privileged -n cnf-ingress -z f5-afm
```

Upgrade the F5Ingress using the new f5ingress helm chart version mentioned in the CNF 2.3.0 tarball.

Note: Before upgrading F5Ingress, ensure that readinessGates.enabled is set to true in values.yaml.

helm upgrade f5ingress tar/f5ingress-<version>.tgz -f <values>.yaml -n namespace

In this example, the Pods will be upgraded using the f5ingress-v14.59.1-0.0.70 Helm chart.

helm upgrade f5ingress tar/f5ingress-v14.59.1-0.0.70.tgz -f overrides_2.3.0_values.yaml -n cnf-ingress

Sample Output:

Release "f5ingress" has been upgraded. Happy Helming!
NAME: f5ingress
LAST DEPLOYED: Thu April 26 11:37:50 2026
NAMESPACE: cnf-ingress
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The F5Ingress Controller has been installed.
TMM debug sidecar is deployed. To access: kubectl exec -it deployment/f5-tmm -c debug -n cnf-ingress -- bash

Note: Need to use extra vlan IP and extra snat IP for the traffic to work seamlessly after upgrade. with maxSurge 1 and maxUnavailable 0.

View the list of pods associated with cnf-ingress namespace/project.
```
kubectl get pods -n cnf-ingress -o wide
```
See the installed releases in cnf-ingress namespace.

In this example, the below command shows the installed releases in cnf-ingress Project.
```
helm list -n cnf-ingress
```
During F5Ingress update process, run the below command to check the status of readinessGates (ConfigurationDone and RoutingDone gates).

In this example, the f5-tmm-7fb766f798-6bgbk is TMM pod.
```
kubectl describe pod f5-tmm-7fb766f798-6bgbk
```
Sample Output:
```
Readiness Gates:
  Type                Status
  ConfigurationDone   True
  RoutingDone         True
```