Upgrading CNFs from v1.3.x to v1.4.0

Overview

The Cloud-Native Network Functions (CNFs) can be upgraded using the typical Helm upgrade process.

This document describes how to upgrade CNFs from v1.3.x to v1.4.0.

Note: 1.3.x refers to any version among 1.3.0, 1.3.1 and 1.3.2 versions.

Requirements

Following are the prerequisites to upgrade the CNFs:

• Ensure v1.3.x of the following are installed:
  • f5-cnf-crds-n6lan
  • f5-cert-manager
  • rabbitmq
  • cwc
  • f5-crdconversion
  • f5-dssm
  • f5-toda-fluentd
  • f5ingress
• A workstation with Helm installed.

Procedures

Upgrading CNFs from 1.3.x to 1.4.0 using F5 Cert Manager

Following are the steps to upgrade the CNFs from v1.3.x to v1.4.0.

Note: Before starting with the upgrade process, delete the applied DoH CR by uisng oc delete dnsapp/doh_name command. If the DoH CR is created from a file, run oc delete -f doh_file.yaml command to delete.

1. Verify the installation STATUS of the CNFs CRDs:

   In this example, the CNFs CRDs are in the default Project.

   helm list -n default
   

   In this example, the release name is f5-cnf-crds.

   NAME          NAMESPACE   REVISION  STATUS     CHART 
   f5-cnf-crds   default     2         deployed   f5-cnf-crds-n6lan-0.177.8-0.0.7.tgz
   

2. Verify the installation STATUS of the Cert Manager Pods:

   oc get pods -n cnf-cert-manager
   

   In this example, the STATUS of f5-cert-manager, f5-cert-manager-cainjector, and f5-cert-manager-webhook Pods is Running.

   NAME                                          READY   STATUS    
   f5-cert-manager-cainjector-5cfbf4ff75-drmh7   1/1     Running   
   f5-cert-manager-cbfc74b4d-kskjx               1/1     Running   
   f5-cert-manager-webhook-58bf4b7b76-bcn4p      1/1     Running   
   

3. Verify the installation STATUS of the RabbitMQ Pod:

   In this example, the RabbitMQ Pod is in the cnf-telemetry Project.

   oc get pods -n cnf-telemetry
   

   RabbitMQ Pod STATUS is Running.

   NAME                              READY   STATUS      
   f5-rabbit-5688f9c8c7-f7d9d        1/1     Running           
   

4. Verify the installation STATUS of the CWC Pod:

   In this example, the CWC Pod is in the cnf-telemetry Project.

   oc get pods -n cnf-telemetry
   

   CWC Pod STATUS is Running.

   NAME                              READY   STATUS      
   f5-cnf-cwc-94bcd64bd-42xdc        1/1     Running           
   

5. Verify the installaton STATUS of the CRD Conversion pod:

   In this example, CRD Conversion Pod is installed in the cnf-crdconversion Project.

   oc get pod -n cnf-crdconversion 
   

   CRD Conversion Pod STATUS is Running

   NAME                                  READY      STATUS       RESTARTS      AGE
   f5-crd-conversion-8478b9y96-asfd1     1/1        Running      0             30s
   

6. Verify the installation STATUS of the dSSM Pod:

   In this example, the dSSM Pod is in the cnf-gateway Project.

   oc get pods -n cnf-gateway
   

   The dSSM Pods STATUS is Running.

   NAME                        READY    STATUS      
   f5-dssm-db-0                3/3      Running   
   f5-dssm-db-1                3/3      Running   
   f5-dssm-db-2                3/3      Running   
   f5-dssm-sentinel-0          3/3      Running   
   f5-dssm-sentinel-1          3/3      Running   
   f5-dssm-sentinel-2          3/3      Running             
   

7. Verify the installation STATUS of the f5-toda-fluentd Pod:

   In this example, the f5-toda-fluentd Pod is in the cnf-gateway Project.

   oc get pods -n cnf-gateway
   

   f5-toda-fluentd Pod STATUS is Running.

   NAME                                  READY   STATUS      
   f5-toda-fluentd-8cf96967b-jxckr       1/1     Running           
   

8. Verify the installation STATUS of the f5-ingress Pod:

   In this example, the f5-ingress Pod is in the cnf-gateway Project.

   oc get pods -n cnf-gateway 
   

   The STATUS of all containers is Running.

   NAME                                    READY   STATUS    RESTARTS    AGE
   f5-afm-797d564479-zwlzw                 2/2     Running   0           2m44s 
   f5-downloader-69474bd866-7j7fv          2/2     Running   0           2m44s
   f5-dssm-db-0                            3/3     Running   0           4m52s
   f5-dssm-db-1                            3/3     Running   0           4m11s
   f5-dssm-db-2                            3/3     Running   0           3m37s
   f5-dssm-sentinel-0                      3/3     Running   0           4m52s
   f5-dssm-sentinel-1                      3/3     Running   0           4m4s
   f5-dssm-sentinel-2                      3/3     Running   0           3m30s
   f5-ipsd-846cb4db8c-pfhqv                2/2     Running   0           2m44s
   f5-tmm-6fc7f98774-b8cxh                 7/7     Running   0           2m44s
   f5-tmm-6fc7f98774-zl4gh                 7/7     Running   0           2m44s
   f5-toda-fluentd-69444c854-zrglf         1/1     Running   0           5m
   f5ingress-f5ingress-79ff4b7c99-hl8qd    5/5     Running   0           2m44s
   otel-collector-7778f8647b-9cnzb         1/1     Running   0           2m44s
   

9. After bringing up the setup, apply the required protocol CRs to run the traffic.

10. Upgrade CNF CRDs using the Helm chart version of CNFs v1.4.0.

    helm upgrade <release> tar/<f5ingress-helmchart-path>.tgz -n <namespace> 
    

    Example:

    helm upgrade f5crds tar/f5-cnf-crds-n6lan-0.177.8-0.0.7.tgz -n default 
    

11. Upgrade the Cert Manager Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade f5-certificate-manager tar/f5-cert-manager-0.22.22-0.0.2.tgz -n cnf-cert-manager -f cm_overrides.yaml
    

12. Upgrade the RabbitMQ Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade cnf-rabbit tar/rabbitmq-0.2.8-0.0.2.tgz -n cnf-telemetry -f rabbitmq_overrides.yaml
    

13. Upgrade the CWC Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade cnf-cwc tar/cwc-0.14.15-0.0.6.tgz -n cnf-telemetry -f cwc_overrides.yaml
    

14. Upgrade the CRD Conversion Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade f5-crd-conversion tar/f5-crdconversion-0.9.4-0.0.3.tgz -n cnf-crdconversion -f crd-conversion-values.yaml
    

15. Upgrade the dSSM Pod using the Helm chart version of CNFs v1.4.0 by adding timeout option:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Note: The default dSSM helm upgrade timeout duration is 5 minutes. Hence, upgrade the dSSM pod by adding timeout 20m option.

    Example:

    helm upgrade f5-dssm tar/f5-dssm-0.67.7-0.0.1.tgz -n cnf-gateway -f dssm-values.yaml --timeout 20m
    

    Note: The following upgrade steps will override the manual changes made to OTEL (f5-ingress-otel-collector-conf, f5-tmstatsd and f5-otel-collector-conf) and TODA (f5-toda-fluentd) config-maps. Before performing the upgrade, keep a backup of the manual changes added in the config-maps and re-apply them post upgrading the helm charts.

16. Upgrade the f5-toda-fluentd Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade f5-fluentd tar/f5-toda-fluentd-1.23.36.tgz -n cnf-gateway -f fluentd-values.yaml
    

17. Upgrade the f5ingress Pod using the Helm chart version of CNFs v1.4.0:

    helm upgrade <release> tar/<helm-chart>.tgz -n <namespace> -f <values>.yaml
    

    Example:

    helm upgrade f5ingress tar/f5ingress-v0.480.0-0.1.30.tgz -n cnf-gateway -f f5ingress_overrides.yaml
    

    Now, the CNFs is upgraded from v1.3.x to v1.4.0.

Upgrading CNFs from 1.3.x to 1.4.0 using Open Source Cert Manager

Following are the steps to upgrade CNFs while using an Open Source Cert Manager

1. Perform the regular upgrade using the steps mentioned in Upgrading CNFs from 1.3.x to 1.4.0 using F5 Cert Manager section.

2. Verify the status of all pods/services. Run the following command:

   oc get pods

   Sample output:

   NAME                                   READY   STATUS    
   f5-afm-5dc59f4c6b-9l5v9                2/2     Running   
   f5-downloader-c959b56f4-v7nf8          1/1     Running   
   f5-dssm-db-0                           3/3     Running   
   f5-dssm-db-1                           3/3     Running   
   f5-dssm-db-2                           3/3     Running   
   f5-dssm-sentinel-0                     3/3     Running   
   f5-dssm-sentinel-1                     3/3     Running   
   f5-dssm-sentinel-2                     3/3     Running   
   f5-ipsd-845d79ffcc-vnjcv               2/2     Running   
   f5-tmm-795c9f96f5-ss4ph                7/7     Running   
   f5-tmm-795c9f96f5-xr7w8                7/7     Running   
   f5-toda-fluentd-78bfcf7757-7792t       1/1     Running   
   f5ingress-f5ingress-855df998fb-jr672   6/6     Running   
   otel-collector-686d9dbccb-blq4t        1/1     Running
   

3. Uninstall the f5-cert-manager. Run the following command:

   helm uninstall -n certmanager f5-cert-manager
   

   Sample output:

   Release "f5-cert-manager" uninstalled.

4. Install the OSS Cert Manager. For more information on how to install OSS Cert Manager, see Open Source Cert Manager and Installing cert-manager .

   Run the following command:

   helm install osscertmgr osscert/cert-manager --version v1.15.0 -f oss-helm-overrides.yaml -n certmanager
   

   Sample output:

   cert-manager v1.15.0 has been deployed successfully!

5. Create an Issuer for OSS cert-manager. To create a cert-manager Issuer, see Issuer. For more information on creating a self-signed issuer and example, see SelfSigned.

   Note: After installing the OSS Cert Manager, add the following OSS references to the overrides.yaml files of all the subsequent CNF component helm upgrades.

   global:
      certmgr:
         external: true
         issuerRef:
            name: your_cluster_issuer_name
            kind: ClusterIssuer
            group: cert-manager.io
   versionValidator:
      name: f5-version-validator
      image:
         repository: "repo.f5.com/images"
   

6. Delete and re-apply the external certificates in OTEL Collectors.

   a. Delete the external certificates and secrets. Verify that they are deleted.

   oc delete certificate external-otelsvr -n cnf-gateway
   oc delete certificate external-f5ingotelsvr -n cnf-gateway
   

   Sample output:

   certificate.cm.f5co.k8s.f5net.com "external-otelsvr" deleted
   certificate.cm.f5co.k8s.f5net.com "external-f5ingotelsvr" deleted
   

   b. Apply the OTEL Collectors CR in f5ingress installing namespace before installing the f5ingress with OTEL enabled.

7. Upgrade the CWC pod.

   helm upgrade cwc -n cwc cwc-0.23.2-0.0.14.tgz -f cwc_overrides.yaml
   

   Sample output:

   Release "cwc" has been upgraded. Happy Helming!

8. Upgrade the CRD Conversion Pod.

   helm upgrade -n crdconverter f5-crdconversion f5-crdconversion-0.13.3-0.0.8.tgz -f f5-crdconversion_overrides.yaml
   

9. Upgrade the RabbitMQ pod.

   helm upgrade rabbitmq -n rabbitmq rabbitmq-0.2.27-0.0.7.tgz -f rabbitmq_overrides.yaml
   

   Sample output:

   Release "rabbitmq" has been upgraded. Happy Helming!

10. Upgrade the dSSM pod.

    helm upgrade f5-dssm -n adi-alpha f5-dssm-0.68.26-0.0.13.tgz -f f5-dssm_overrides.yaml
    

    Sample output:

    Release "f5-dssm" has been upgraded. Happy Helming!

11. Upgrade the f5-toda-fluentd pod.

    helm upgrade f5-toda-fluentd -n adi-alpha f5-toda-fluentd-1.27.7-0.0.12.tgz -f f5-toda-fluentd_overrides.yaml
    

    Sample output:

    Release `f5-toda-fluentd` has been upgraded. Happy Helming!

12. Upgrade the f5ingress pod.

    helm upgrade f5ingress -n f5ingress f5ingress-0.542.0-0.0.122.tgz -f f5ingress_overrides.yaml
    

    Sample output:

    Release `f5ingress` has been upgraded. Happy Helming!

13. Upgrade the CRD bundle.

    helm upgrade f5-cnf-crds -n default f5-cnf-crds-n6lan-0.177.8-0.0.10.tgz -f f5-cnf-crds-n6lan_overrides.yaml
    

    Sample output:

    Release `f5-cnf-crds` has been upgraded. Happy Helming!

CNFs Upgrade Support - Best Practices

Upgrade

This section describes the procedure to upgrade the CNFs from an older version to a new version while ensuring least possible disruptions.

Note: Do not upgrade CRDs in first pass until the upgraded deployment is stable.

Sample scenario:

Upgrade from CNFs version 1.3.x to 1.4.0.

1. Do a helm upgrade from version 1.3.x to 1.4.0 (do not perform the CRD bundle upgrade). Since it is an upgrade from 1.3.x to 1.4.0 version, all the CRs will be compatible with 1.4.0 version and the upgrade will be successful even without the CRD bundle.

   Sample command:

   Upgrade only f5ingress 1.4.0

   helm upgrade <deployment-name> <1.4.0-f5ingress-helmchart-path.tgz> -f <values.yaml>

   Example:

   helm upgrade f5ingress tar/f5ingress-v0.480.0-0.1.30.tgz -n cnf-gateway -f f5ingress_overrides.yaml.

2. If the upgrade is successful, and the admin is satisfied with the traffic statistics, then the admin can apply the CRD bundle update explicitly after some time and start using the new CRs.

   Sample command:

   To upgrade CRDs to v1.4.0, run the following command:

   Helm upgrade <crd-deployment name> <1.4.0 crd-tarball path>

   Example:

   helm upgrade f5crds tar/f5-cnf-crds-n6lan-0.161.0-0.1.2.tgz -n default

3. The steps mentioned earlier in this procedure will catch most of the common upgrade issues and are least disruptive.

4. Once the upgrade is successful and the admin has applied the CRD bundle, in such a case, you have to delete all the F5 CRDs before changing the CNFs version. This is similar to a fresh install and there is no workaround available as of now to avoid deleting the F5 CRDs.

   To manually delete all the F5 CRs and CRDs installed on the cluster, run the following command:

   oc get crds -oname | grep f5-big  | xargs oc delete

Feedback

Provide feedback to improve this document by emailing cnfdocs@f5.com.

Supplemental

• Using Helm