Install BIG-IP Next for Kubernetes on NVIDIA BlueField-3 DPU

Prepare the CNE core and install F5 Lifecycle Operator

Create the namespace for the CNE core and create an image pull secret

  1. Create the namespace for the CNE core:

    kubectl create namespace f5-cne-core
    
  2. Create an image pull secret to allow Kubernetes to pull container images from F5 Artifact Registry:

    kubectl apply -f far-pull-secret.yaml \
      --namespace f5-cne-core
    

Download and extract the archive file for the f5-cert-gen chart

  1. Log in to F5 Artifact Registry with Helm:

    cat cne_pull_64.json | helm registry login \
        --username _json_key_base64 \
        --password-stdin repo.f5.com
    
  2. Download the archive file for the f5-cert-gen chart:

    helm pull oci://repo.f5.com/utils/f5-cert-gen \
      --version <chartVersionFromReleaseManifest>
    

    Note

    Replace <chartVersionFromReleaseManifest> with the version of utils/f5-cert-gen specified in the release manifest that you downloaded earlier (f5-bigip-k8s-manifest-<version>.yaml).

  3. Extract the archive file:

    tar zxvf f5-cert-gen-<chartVersionFromReleaseManifest>.tgz
    

    Extracting the archive file creates a directory named cert-gen that contains the gen_cert.sh script.

Create the TLS credentials required to authenticate with the F5 CWC API

  1. Generate the certificates and secrets.

    sh cert-gen/gen_cert.sh \
      -s=api-server \
      -a=f5-spk-cwc.f5-cne-core.svc.cluster.local \
      -n=1
    

    The script creates the following manifests:

    • cwc-license-certs.yaml

    • cwc-license-client-certs.yaml

  2. Create the secrets:

    kubectl create \
      -f cwc-license-certs.yaml \
      -f cwc-license-client-certs.yaml \
      --namespace f5-cne-core
    

    The command creates the following secrets in the f5-cne-core namespace:

    • cwc-license-certs

    • cwc-license-client-certs

Decide which licensing mode to use

Use this table to help you decide which licensing mode is best for you:

Licensing mode

Description

Connected

F5 Cluster-Wide Controller (CWC) connects directly to the F5 licensing server via the public internet.

Disconnected

Your cluster is not connected to the public internet. You must manually perform licensing for your cluster.

Download your license key from MyF5

  1. Log in to MyF5.

  2. Go to My Products & Plans > Subscriptions to see your active subscriptions.

  3. Find your BIG-IP Next SPK subscription, and select the Subscription ID for details.

  4. Download the JSON Web Token (License key) file.

    Example

    SFA-1515255-F5-subscription-2.jwt

Create an override values file for F5 Lifecycle Operator

  1. Create a file to use as the override values file for F5 Lifecycle Operator (FLO):

    touch f5-lifecycle-operator-override-values.yaml
    
  2. Open the file and add the following information to it:

    global:
      certmgr:
        clusterIssuer: arm-ca-cluster-issuer
      imagePullSecrets:
      - name: far-pull-secret
    
    ServiceIPFamily: ipv4
    
    image:
      pullPolicy: IfNotPresent
    
    fluentbit_sidecar:
      fluentd:
        host: f5-toda-fluentd.f5-cne-core.svc.cluster.local.
      resources:
        limits:
          cpu: 500m
          memory: 1Gi
        requests:
          cpu: 300m
          memory: 512Mi
    
    # Resources for the f5-lifecycle-operator container.
    resources:
      limits:
        cpu: "0.5"
        memory: "512Mi"
      requests:
        cpu: "0.25"
        memory: "256Mi"
    
    affinity: {}
    
    tolerations: []
    
    # For license related configuration
    license:
      operationMode: "connected"
      # JWT license key used to activate with the F5 licensing server.
      jwt: "<licenseKey>"
      friendlyName: "CNE Cluster"
      teemCertUrl: "https://product.apis.f5.com/ee/v1"
      teemEntitlementUrl: "https://product-s.apis.f5.com/ee/v1"
      teemInitialConfigUrl: "https://product-s.apis.f5.com/ee/v1"
      kid: "v1"
      alg: "RS512"
      kty: "RSA"
      modulus: "wgqDv-fuebdh_gV3wN8voRGcHGDo4YekYT78U2x-gAgxWDFFP4uIpQk9d_Hszevyr78xgFBD7RnR4FeWu7R62L1DnEEbrQYEjNI1quLtcfI5wvfdMLeyv5ZXe4-Uu21lwhvRtJCyzNRW797NPaVsMAzrcWwslHZdrljgbLaf3UX90ITyqd9MFgb7jNmRLjdzZLDqQjyYJTq2AOaAoagX0pqKkBWGlgNbiFAuV2RMUhis9Ei5NSfiD5I5ntsvp_Q_XxcJrYrJOs9uaXkeTMXa58JUeX7Tt6zLA6Ju1wxClYMyNWj4u5dG8u2yhCrucJ2_IvTs4A1BSOfAWW5TwS8BBYuY9H_OXgoqyYybeOU5YOHmszaDXfMs7FFZkjAFIfr95bprH-HW1cwREQ_eqkYxi9XmA3KCNQGLhZ_i9gRTu8MlGwYzckeJG_NOyPIa7WRw7zFe7RHcchqaCSjpWC4u-GSZ92uf-EUw2Db7OX4J8VcD_3cvCfCEzfsrSaTkLZvlkYLC-3x4eM7B2GiM7SA8l4u0DK0nrScLsOR5njd2IUUB38K39Jq-tlD5PoqLT6u0AE4IJtWN-S6uFhqzdUvExBYK19ZhTbeMmOEkMALlEPhsNUnOSXGnocxlsRYldwfCoNtpVYxlXrSnbHkuPlK8Q27yi1wO9r3RwO_OxQnUvXk"
      exponent: "AQAB"
      x5c:
      - "MIIFpzCCBI+gAwIBAgIQcC/1AFczGqUAmtLfiN4SvzANBgkqhkiG9w0BAQsFADCBpzELMAkGA1UEBhMCVVMxEzARBgNVBAgMCldhc2hpbmd0b24xGjAYBgNVBAoMEUY1IE5ldHdvcmtzLCBJbmMuMR4wHAYDVQQLDBVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxNTAzBgNVBAMMLEY1IFBSRCBJc3N1aW5nIENlcnRpZmljYXRlIEF1dGhvcml0eSBURUVNIFYxMRAwDgYDVQQHDAdTZWF0dGxlMB4XDTIyMDEyNTE2MzYwOVoXDTI3MDEyNDE3MzYwOVowgYExCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApXYXNoaW5ndG9uMRAwDgYDVQQHDAdTZWF0dGxlMRowGAYDVQQKDBFGNSBOZXR3b3JrcywgSW5jLjENMAsGA1UECwwEVEVFTTEgMB4GA1UEAwwXRjUgUFJEIFRFRU0gSldUIEF1dGggdjEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDCCoO/5+55t2H+BXfA3y+hEZwcYOjhh6RhPvxTbH6ACDFYMUU/i4ilCT138ezN6/KvvzGAUEPtGdHgV5a7tHrYvUOcQRutBgSM0jWq4u1x8jnC990wt7K/lld7j5S7bWXCG9G0kLLM1Fbv3s09pWwwDOtxbCyUdl2uWOBstp/dRf3QhPKp30wWBvuM2ZEuN3NksOpCPJglOrYA5oChqBfSmoqQFYaWA1uIUC5XZExSGKz0SLk1J+IPkjme2y+n9D9fFwmtisk6z25peR5MxdrnwlR5ftO3rMsDom7XDEKVgzI1aPi7l0by7bKEKu5wnb8i9OzgDUFI58BZblPBLwEFi5j0f85eCirJjJt45Tlg4eazNoNd8yzsUVmSMAUh+v3lumsf4dbVzBERD96qRjGL1eYDcoI1AYuFn+L2BFO7wyUbBjNyR4kb807I8hrtZHDvMV7tEdxyGpoJKOlYLi74ZJn3a5/4RTDYNvs5fgnxVwP/dy8J8ITN+ytJpOQtm+WRgsL7fHh4zsHYaIztIDyXi7QMrSetJwuw5HmeN3YhRQHfwrf0mr62UPk+iotPq7QATggm1Y35Lq4WGrN1S8TEFgrX1mFNt4yY4SQwAuUQ+Gw1Sc5JcaehzGWxFiV3B8Kg22lVjGVetKdseS4+UrxDbvKLXA72vdHA787FCdS9eQIDAQABo4HyMIHvMAkGA1UdEwQCMAAwHwYDVR0jBBgwFoAUg6RSCVfoA+ncTgkMv61aIxsPLegwHQYDVR0OBBYEFCKHAv7lwN4DEyUCOp0XHcWZ8QySMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwcwYDVR0fBGwwajBooGagZIZiaHR0cDovL2NybC10ZWVtLXByZC1vcmUtZjUuczMudXMtd2VzdC0yLmFtYXpvbmF3cy5jb20vY3JsLzJlNmVhYWI3LTFmNDQtNGRmYS05MTY3LWQ2MjZjOTAzY2M2Zi5jcmwwDQYJKoZIhvcNAQELBQADggEBAGwFGNJVigk6rrJRy5SVUYFR1LPEE/LpdpaFfZjbiviG0LIzu3TA6sbyj6+KFbdQX+tvK46JvDmdTDow23gvqdujPMBcwIGVstg1PYYae8rq29iu3mmC5Y/bWJSYD34hxkUE8k3z3r7aCuUkle3jOwYMgQqVqT/CmdbOVlgBGW+qL2+jD0kBCarJUH6Ckb6X5rFQB/X6bisIxuA6ubYsIsDiPYlt87xScHlIjH2hVuDU/QAXpxL7SvKCsLU8GDCpQqHjJusaD48o2/zDGalqVpjZ9f+McC7fj/DAHidzTvJ44GTxQ+5yeSput9rcpkTwHmJ2TAqDAWZ9HXC0X/1pJ8o="
      - "MIIE/jCCAuagAwIBAgIBAjANBgkqhkiG9w0BAQsFADB1MQswCQYDVQQGEwJVUzELMAkGA1UECBMCV0ExGTAXBgNVBAoTEEY1IE5ldHdvcmtzLCBJbmMxPjA8BgNVBAMTNUY1IFBSRCBJbnRlcm1lZGlhdGUgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IEY1QVBJLVBNIFYyMB4XDTE5MDEwNDE3NTIyNVoXDTI5MDEwMTE3NTIyNVowgacxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApXYXNoaW5ndG9uMRowGAYDVQQKDBFGNSBOZXR3b3JrcywgSW5jLjEeMBwGA1UECwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MTUwMwYDVQQDDCxGNSBQUkQgSXNzdWluZyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgVEVFTSBWMTEQMA4GA1UEBwwHU2VhdHRsZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALXsiCog/VlPmHOcPNC0Q5hAm5fsY7IkdBqaDJ/TO+XRsyMQbZBFgbG8MueW2d6jVVYKoJeT+5q8HltzF31yUjWMRAznJGyawN3Rvb0+8ZeHeIJRkUQHjXvo7Mx3tUwhkcPVxYZB0XGjA1jZakminP7x/uOKhZgzafDT165hGpFbtTjBcTmtLHKIWiX27toNbDr4J9DMtpeo0MVJYgaaZVsV0BjL+6R3pYiWhJlomJVtrlFMBL6o9LgzlBPq7i5fiAjkkuJC3f0am/HP9PzbvPjmpL70/Z8el6w48UpTuYbG3gYLILVoRByTyGn8QzeKHcqsMvXkwGIwPTyhcK+gQ/sCAwEAAaNmMGQwHQYDVR0OBBYEFIOkUglX6APp3E4JDL+tWiMbDy3oMB8GA1UdIwQYMBaAFFqYykyPkWydzv5wl+VsJZ88cOeMMBIGA1UdEwEB/wQIMAYBAf8CAQEwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQASUz5XADaWb4gIjcdOCrqfCvq95CaIjM4TLn4pMaGL9MaRW498CzsVNqjaJKe5t3De6xONMheocnvPMV1V3JZ6olaDcseDPTo3V/Gu+PVAv0/qOpmbJmipM5yHcY6bM6Ek+EQOAjHjs4bV+IrDyorUIHtHfSzFhKSmIiBafyuCsbuG3z9B7ty7Rm9jjayLSLRmEbpN+IkqKQGNCWlVHlEwQ6NY+kIhXAfHQf6JpKLsw36ukkvpQGGb46sQsXI2Bh94yM7xxQs0baXyHSCck6h9Az2FUQxRjqDQCANjLiMe2qclWCuPS4o/i792mWC/AQMXj+ofH3vLaPjSm5eETeBpcSOMrczY9V75Tn+WN7h/fpIHyiToZ3IAHoncDuaXUCTvJ6uMZPLjb8hEX9aqYwvPwYI9IADcqdx3SGd69JLmk7o+P7pcX0rlAttMQC8WOJfjUlex9jjSLn+6ArPa94+Agckh09r5M8X1GOMpQatXr/QZ76qaBEaq6QuKJBT7j4vvMDbv9AtU87+v478Joiv2Br74V3ZtCrneE8OT1AjeaZ6++8bxNIuvvG1qYq48sZv4RIMw+uodLoyNUOUqgTx/N2I6Rrw9ytxQV3v0smZTLczWx3dgACZnc5ngMLiaxRVtPcLSxogTEjpCKOjk4wk8f10h99Hy+Vl3eA2laE2lFA=="
      - "MIIHHjCCBQagAwIBAgIJAN0Ej1LicyHeMA0GCSqGSIb3DQEBCwUAMIG6MRkwFwYDVQQKExBGNSBOZXR3b3JrcywgSW5jMRowGAYDVQQLExFJbnRlcm5hbCBVc2UgT25seTEdMBsGCSqGSIb3DQEJARYOcm9vdEBsb2NhbGhvc3QxEDAOBgNVBAcTB1NlYXR0bGUxCzAJBgNVBAgTAldBMQswCQYDVQQGEwJVUzE2MDQGA1UEAxMtRjUgUFJEIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IEY1QVBJLVBSIFYyMB4XDTIyMDgxNzAxNTExNFoXDTMyMDgxNDAxNTExNFowgboxGTAXBgNVBAoTEEY1IE5ldHdvcmtzLCBJbmMxGjAYBgNVBAsTEUludGVybmFsIFVzZSBPbmx5MR0wGwYJKoZIhvcNAQkBFg5yb290QGxvY2FsaG9zdDEQMA4GA1UEBxMHU2VhdHRsZTELMAkGA1UECBMCV0ExCzAJBgNVBAYTAlVTMTYwNAYDVQQDEy1GNSBQUkQgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgRjVBUEktUFIgVjIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDL56chAk5eBWVsaJ2j2/2cBnxROaqRgVMWNXYwsEKL9StlMoPXGmfO3mDDsFfePCBKbQKseYrvDWf/caQtE64nvBcjkQpDReG+1wA/mpRXm38Y/0779roksExa+703zcd5tzqNfI7onvnPWfBgDlcZ6rUBlYKezbDRSHj7/T8b/y09+YN1apbzFypfBetZ5FnYkIxPWQvIytKw0CU/Un3qeAWwSKsk8nqQeotC9H9dJzriccern/sbJZEf390x9o7OXRb6D01Yy8RBrNdqo2EN9f/zHuITtX0+BJJodDCleEQkQhGZCt6/jKqNRToYJ45sjc9nMgbd4lBUqY4CxLJjCCe9gs46TwpghO3g4s4PppwkFnP42ZBpYSsBhLxmJl6H0+zP1pFOeXRYWW05SgiW0jQZ3Ucp+FNlob6i2rhD71jHFmi6LEqhSrOsgl+hwhALvp0YqryJ7fdgLAxjcN8pglvfy8CwZFH7AhaSqUYsSDKeytbM77PkwdtubnhHOBYuXy13f/aDX1hmXxqLGr0DaVnQmazLhMJb2ZcXdrCiLGImlyvN1+KwdI0SqmWhi7fCZHvk0x9i9xDc0dRLcYS92jgvkh8fpmZICTk0aGAps+CF8I7ndPVtF/2UPhaZq18p4PQAwWu1LYxKfygcfZuytMSPY+3J9u0R2UAXi2DzlQIDAQABo4IBIzCCAR8wDAYDVR0TBAUwAwEB/zAdBgNVHQ4EFgQUNyeBj+lAqBvokj8drl3gs60xD5Qwge8GA1UdIwSB5zCB5IAUNyeBj+lAqBvokj8drl3gs60xD5ShgcCkgb0wgboxGTAXBgNVBAoTEEY1IE5ldHdvcmtzLCBJbmMxGjAYBgNVBAsTEUludGVybmFsIFVzZSBPbmx5MR0wGwYJKoZIhvcNAQkBFg5yb290QGxvY2FsaG9zdDEQMA4GA1UEBxMHU2VhdHRsZTELMAkGA1UECBMCV0ExCzAJBgNVBAYTAlVTMTYwNAYDVQQDEy1GNSBQUkQgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgRjVBUEktUFIgVjKCCQDdBI9S4nMh3jANBgkqhkiG9w0BAQsFAAOCAgEAZyGOh2AAoEEVr0OT4l9IEXB136Hoo1s1MMpgHADFMks6DGHmU6ldKncYqPVK/3DeAnrF+/a+/v5nlOShp0pbABaeHg7ZDXA88Ho2dObOM26TgUOWWjnsuKXHuih7YbMoh9tAQ5hYv1b3ugSiviysYHDCPyhdfRxkj11KB7GCYJH839UcmTUu/V9ERwYhMz8CY5rOi93xim5jlJrsdlrXDK/CXo5olCT7zVIkK8L05eiAoAXGzErO2/3TZQQcR3cjST5+WBC5e+56oUkQ+qund4v7qkelyrDVBYAT0KqCMY+79V5PUwaQ9Y7iE4pr/BIxtf/HM2pTJr+6fLJ3UUBZtXREiqxLALJY1lIRimiaqkEEugb7NOT2ThHe9Jd3oam6wqzgleSj8k4dN23/+59LXFskY821VvtPGSlgMir8sSZJ9cKB5UWvljZ89KZl8KAw8GvhfsFvfWQzL2JVGyDj9jsq23mixDbbNHgFx2BTmkinDJVj/lPuoKACzpZrYAxBKajUUUcjubEz3ZyAUoV8UaJ8rRzwcKoK4hIwS7UXciE0k2WJ3HnSEI/R8ptHPm1s5qLzu7ES4dxy4sOEOehr1YffW3V5Qw1Nk9/9MHhHwWAzK+jE4KcPPd9bkVLWwhGFx3IEpBeauOi3tSnAXj4LU6b3QN7ulUFK780Clu9hJB8="
      - "MIIGETCCA/mgAwIBAgIBGDANBgkqhkiG9w0BAQsFADCBujEZMBcGA1UEChMQRjUgTmV0d29ya3MsIEluYzEaMBgGA1UECxMRSW50ZXJuYWwgVXNlIE9ubHkxHTAbBgkqhkiG9w0BCQEWDnJvb3RAbG9jYWxob3N0MRAwDgYDVQQHEwdTZWF0dGxlMQswCQYDVQQIEwJXQTELMAkGA1UEBhMCVVMxNjA0BgNVBAMTLUY1IFBSRCBSb290IENlcnRpZmljYXRlIEF1dGhvcml0eSBGNUFQSS1QUiBWMjAeFw0yMjA4MTcxNjAxMjBaFw0zMjA4MTQxNjAxMjBaMHUxCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJXQTEZMBcGA1UECgwQRjUgTmV0d29ya3MsIEluYzE+MDwGA1UEAww1RjUgUFJEIEludGVybWVkaWF0ZSBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgRjVBUEktUE0gVjIwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC87P9OSZd3BG6JpelO6fEuDxT5NvtNIQANSGwL3889yHRPduKx1JdZBl1BIUp6ZmEUSwt2Fp/TUDukgiawN3L7FA9w4xLOGFoiO0qLEPANygEbA4MdfMNfh8+ceNOXkIO1H3KHRixC8q+edaQHjCWju3Vrz9zYLGaDbSTD71U/volviFzr/fVQBPJqRoW51z8xO7SlM0U6bBr4LeXQtzJUrPgzrApCDQJ5081LEGdrWAZuHi2YVCUgwDgxWSJRPHCGpvXBjbsjz6ZncO0Be9NWgU0Eljg95hj2qyujMK+BUNO0MTAM3FlyyVI2QhqWdPgZUaZzkGlY8snML62ypdjt0jKTJmXRZLY//ZSve4j6EE0Lk7uAfi7O902mSPuQmUnvEX9psXQruq6M4wiChCEER04YTXNShPxfjmQk6Xa8a/NP94aQGQqGqpY5bqMNNjfJEU0Va8lh61FdUltB9sCX+xq0o+iXd92eJqXeZsXTdJld2cFW6N0hI7SlpLcHdMJNbuLDUk3JbcgflUU/gSXARYImQVvSG7s0sdvCw66zrgGZOtkI0oSRmlPofJmxkJXBl4Eo4xwe6mwtsQLZ0XeRePKbs2gcQhHAWJqzRinsC3TneaIb7rW+0c2grPn+gcd8jUI3zbc62Y8wz4YuLkWGrZMttvvGWYoXPkDaGba5ywIDAQABo2YwZDAdBgNVHQ4EFgQUWpjKTI+RbJ3O/nCX5Wwlnzxw54wwHwYDVR0jBBgwFoAUNyeBj+lAqBvokj8drl3gs60xD5QwEgYDVR0TAQH/BAgwBgEB/wIBATAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQELBQADggIBAFzxxIcmKXmwBeiT73s8NdYjwsfhXn0LYrD5DgYYs5/SmPZ/M2CsMkVOGXgYQmqXxQl9CZWAn4vEnpGW9LMi2buPFHgEfSLxw2ESNnxu5x+GForjVebtexeZl83dEkNBfKshL+XuV/ZRYilRpDYA9hionMBwVZxmNH+oMT1I7GIxGZlTD6Qko0K+KKmqua2GebIO+6L56Bht/791BrARXDjOINgyzJAYYahpE3C5XQWj8QoZjXP+iA4LyfBJO2MXcpOKnGJRNDIFMJnOKvcbJenPOy3jH2fOA9Dz7FxZKffxV/wss/W03xrASrQSWnrZl1vBkRVuJTohCodJBQ2TyzjzoUhx0biwQDJgOpSGs5bvu0CnVX99CpCshHS6cU66ry6tFIcyXcZi+EJ37E0+27WDWGf8HuRBa+DfYDGJw7rKIjaMaCo+nijyBomvUMLp+pLFmPQhS7IdjjSxtg90G6i+vA13t79b7NmaTFSogdJA+wNtVuMm2/dZGk85NOQYSepArKc88BfOXjmfRsO5ANnFbDXZwz59WksfXjwItXGRsdOasG1ZvnxbGTB+PdZ5sqIpVFFG8ioSvhmM3kj/4nUJ7H5N4KjgXY9krnTiMHYXtbm2ckXDahAq/9s8CGG6/VsdBkbO3007A/qJtXQMxeyfVdY2Chjs2X2VqdkWYd18"
      use: "sig"
    
  3. Replace <licenseKey> with the license key (JSON Web Token, or JWT) you downloaded from MyF5.

    Note

    Make sure the license key is enclosed in quotation marks.

  4. If you want to use disconnected mode instead of connected mode as your licensing mode, change the value of .license.operationMode to disconnected.

    For help deciding which mode to use, see Decide which licensing mode to use.

  5. Save and close the file.

Install F5 Lifecycle Operator

  1. Install F5 Lifecycle Operator (FLO):

    helm install f5-lifecycle-operator \
      oci://repo.f5.com/charts/f5-lifecycle-operator \
      --version <chartVersionFromReleaseManifest> \
      --namespace f5-cne-core \
      --values f5-lifecycle-operator-override-values.yaml
    

    Note

    Replace <chartVersionFromReleaseManifest> with the version of charts/f5-lifecycle-operator specified in the release manifest that you downloaded earlier (f5-bigip-k8s-manifest-<version>.yaml).

  2. Verify F5 Lifecycle Operator is running:

    kubectl get pods --namespace f5-cne-core | grep f5-lifecycle-operator
    

    Expected output:

    f5-lifecycle-operator-xxxx-xxxx   2/2   Running   0   1m
    

Install the CNE instance

Create the namespace for the CNE instance and create an image pull secret

  1. Set the CNE_INSTANCE_NAMESPACE environment variable to the name you want to give to the namespace for the CNE instance.

    Example

    export CNE_INSTANCE_NAMESPACE=f5-bnk-instance
    
  2. Create the namespace for the CNE instance:

    kubectl create namespace $CNE_INSTANCE_NAMESPACE
    
  3. Create an image pull secret to allow Kubernetes to pull container images from F5 Artifact Registry:

    kubectl apply -f far-pull-secret.yaml \
      --namespace $CNE_INSTANCE_NAMESPACE
    

Create MULTUS network-attachment definitions

  1. Create a file to use as the manifest for two network-attachment definitions:

    touch net-attach-def.yaml
    
  2. Open the file in a text editor.

  3. Add the following information to the file:

    apiVersion: "k8s.cni.cncf.io/v1"
    kind: NetworkAttachmentDefinition
    metadata:
        name: sf-external  # If you change this value, change it below too.
        annotations:
            k8s.v1.cni.cncf.io/resourceName: nvidia.com/bf3_p0_sf
    spec:
        config: '{
        "type": "sf",
        "cniVersion": "0.3.1",
        "name": "sf-external"
    }'
    
    ---
    apiVersion: "k8s.cni.cncf.io/v1"
    kind: NetworkAttachmentDefinition
    metadata:
        name: sf-internal  # If you change this value, change it below too.
        annotations:
            k8s.v1.cni.cncf.io/resourceName: nvidia.com/bf3_p1_sf
    spec:
        config: '{
        "type": "sf",
        "cniVersion": "0.3.1",
        "name": "sf-internal"
    }'
    
  4. Save and close the file.

  5. Create the network-attachment definitions:

    kubectl apply -f net-attach-def.yaml --namespace $CNE_INSTANCE_NAMESPACE
    

Create the certificates for the Prometheus-scraping endpoint for the CNE instance

  1. Create a file to use as the manifest for the certificates:

    touch otel-certs.yaml
    
  2. Open the file in a text editor and add the following information to the file:

    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
        name: external-otelsvr
    spec:
        commonName: otel.cluster.local
        secretName: external-otelsvr-secret
        issuerRef:
            name: arm-ca-cluster-issuer
            kind: ClusterIssuer
        duration: 8640h
        privateKey:
            rotationPolicy: Always
            encoding: PKCS1
            algorithm: RSA
            size: 4096
        revisionHistoryLimit: 10
    ---
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
        name: external-f5ingotelsvr
    spec:
        commonName: otel.cluster.local
        secretName: external-f5ingotelsvr-secret
        issuerRef:
            name: arm-ca-cluster-issuer
            kind: ClusterIssuer
        duration: 8640h
        privateKey:
            rotationPolicy: Always
            encoding: PKCS1
            algorithm: RSA
            size: 4096
        revisionHistoryLimit: 10
    

    Note

    Update spec.issuerRef.name and spec.issuerRef.kind to match your cert-manager setup. For details, see Configure cert-manager.

  3. Save and close the file.

  4. Create the certificates in the CNE-instance namespace:

    kubectl apply -f otel-certs.yaml \
      --namespace $CNE_INSTANCE_NAMESPACE
    

Install the CNE-instance workloads

Note

Before installing the workloads for a CNE instance, F5 Lifecycle Operator automatically installs the CNE-core workloads and custom resource definitions (CRDs) if they don’t already exist.

  1. Create a file to use as the manifest for the CNE-instance workloads:

    touch f5-bnk-instance.yaml
    
  2. Open the file in a text editor and add the following information to the file:

    apiVersion: k8s.f5.com/v1
    kind: CNEInstance
    metadata:
      name: f5-bnk-instance
    spec:
      product:
        gatewayAPI: true
        type: BNK
      manifestVersion: <releaseManifestVersion>
      wholeCluster: true
      dpu:
        enabled: true
      telemetry:
        loggingSubsystem:
          enabled: true
        metricSubsystem:
          enabled: true
      certificate:
        clusterIssuer: arm-ca-cluster-issuer
      deploymentSize: Large
      registry:
        uri: repo.f5.com
        imagePullSecrets:
        - name: far-pull-secret
        imagePullPolicy: IfNotPresent
      networkAttachments:
      - sf-external
      - sf-internal
      dynamicRouting:
        enabled: true
      # AFM
      firewallACL:
        enabled: true
      pseudoCNI:
        enabled: true
      # Core dump files
      coreCollection:
        enabled: true
      advanced:
        envDiscovery:
          enabled: false 
          stopOnFail: true
          runAfterSuccess: true
        cneController:
          env:
          - name: TMM_DEFAULT_MTU
            value: "9000"
        demoMode:
          enabled: false
        maintenanceMode:
          enabled: false
        tmm:
          env:
          - name: TMM_CALICO_ROUTER
            value: default
          - name: TMM_DEFAULT_MTU
            value: "9000"
          - name: ZEBOS_STATE
            value: legacy
        pseudoCNI:
          env:
          - name: EXCLUDE_CIDR
            value: "10.19.1.80/16,bb10::19:1:80/96"
    
  3. Replace <releaseManifestVersion> with the release-manifest version for your version of F5 BIG-IP Cloud-Native Edition (see Download the Release Manifest File).

  4. Save and close the file.

  5. Trigger F5 Lifecycle Operator to install the CNE-instance workloads:

    kubectl apply -f f5-bnk-instance.yaml --namespace $CNE_INSTANCE_NAMESPACE
    
  6. Monitor the F5 Lifecycle Operator deployment progress by checking the status of each component CR:

    kubectl describe cneinstance f5-bnk-instance --namespace $CNE_INSTANCE_NAMESPACE
    

    Look for the following condition types (in .Status.Conditions[].Type):

    Condition type

    Value

    Meaning

    Accepted

    True

    CR received by F5 Lifecycle Operator

    Reconciled

    True

    Config applied to component

    Available

    True

    Component running and healthy

Verify the installation

  1. Check that all CNE-core pods are running:

    kubectl get pods --namespace f5-cne-core -o wide
    

    Expected pods:

    Pod

    Containers

    Description

    f5-coremond-xxxx

    2/2 (one replica per node)

    F5 core-file collector

    f5-crdconversion-xxxx

    2/2

    F5 CRD Conversion

    f5-ipam-ctlr-xxxx

    2/2

    F5 IPAM controller

    f5-lifecycle-operator-xxxx

    2/2

    F5 Lifecycle Operator

    f5-observer-x

    2/2

    F5 TODA metrics observer

    f5-observer-operator-xxxx

    2/2

    F5 TODA metrics observer operator

    f5-observer-receiver-x

    2/2

    F5 TODA metrics observer receiver

    f5-rabbit-xxxx

    2/2

    RabbitMQ

    f5-spk-csrc-xxxx

    2/2

    Source routing controller

    f5-spk-cwc-xxxx

    2/2

    F5 Cluster-Wide Controller (CWC)

    f5-toda-fluentd-xxxx

    1/1

    F5 TODA Fluentd

    otel-collector-xxxx

    1/1

    OpenTelemetry collector

  2. Check that all CNE-instance pods are running.

    kubectl get pods --namespace $CNE_INSTANCE_NAMESPACE -o wide
    

    Expected pods:

    Pod

    Containers

    Description

    f5-afm-xxxx

    2/2

    F5 Advanced Firewall Manager

    f5-cne-controller-xxxx

    1/1 to 5/5

    F5 CNE-instance controller

    f5-downloader-xxxx

    2/2

    f5-dssm-db-x

    3/3

    F5 dSSM database (3 replicas)

    f5-dssm-sentinel-x

    3/3

    F5 dSSM sentinel (3 replicas)

    f5-tmm-xxxx

    3/3 to 7/7

    F5 TMM daemon set (one pod per DPU node)

  3. Verify that TMM is running on the DPU nodes:

    kubectl get pods -l app=f5-tmm -o wide --namespace $CNE_INSTANCE_NAMESPACE
    

    Note

    Verify that the NODE column shows only DPU node names.

  4. Verify the CRD installation by F5 Lifecycle Operator:

    kubectl get crds | grep f5
    

    Expected CRDs include: f5spkvlans, f5spkstaticroutes, f5spkegresses, f5spksnatpools, f5bigfwpolicies, f5bigfwrulelists, and others.


License your cluster

Prepare to communicate with the F5 CWC API

  1. Extract the F5 Cluster-Wide Controller (CWC) API client key from the cwc-license-client-certs secret and store it in f5-cwc-api-client.key:

    kubectl get secret cwc-license-client-certs \
      --namespace f5-cne-core -o json | \
      jq -r '.data["client-key"]' | \
      base64 -d > f5-cwc-api-client.key
    
  2. Extract the F5 CWC API client certificate from the cwc-license-client-certs secret and store it in f5-cwc-api-client.crt:

    kubectl get secret cwc-license-client-certs \
      --namespace f5-cne-core -o json | \
      jq -r '.data["client-cert"]' | \
      base64 -d > f5-cwc-api-client.crt
    
  3. Extract the F5 CWC API root certificate from the cwc-license-client-certs secret and store it in f5-cwc-api-root.crt:

    kubectl get secret cwc-license-client-certs \
      --namespace f5-cne-core -o json | \
      jq -r '.data["ca-root-cert"]' | \
      base64 -d > f5-cwc-api-root.crt
    
  4. Use port forwarding to create a secure tunnel between your computer and the F5 CWC API:

    kubectl port-forward service/f5-spk-cwc 38081 \
      --namespace f5-cne-core > /dev/null & disown; \
      CWC_API_PORT_FORWARD_PID=$!
    

    Note

    This command:

    • Forwards port 38081 on your computer to the target port associated with port 38081 of a pod in the f5-spk-cwc service in the f5-cne-core namespace (the F5 CWC API)

    • Suppresses normal output (> /dev/null)

    • Runs the process in the background (&)

    • Detaches the process from the shell to avoid creating an associated job (disown;)

    • Saves the process ID so you can later terminate the process

(Disconnected mode only) License your cluster manually

  1. Using the secure tunnel, retrieve your cluster’s license information using the F5 CWC API:

    curl -s \
      --cacert f5-cwc-api-root.crt \
      --cert f5-cwc-api-client.crt \
      --key f5-cwc-api-client.key \
      --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \
      -H "Authorization: Bearer $(
        kubectl get secret cwc-auth-token -n f5-cne-core \
          -o jsonpath='{.data.token}' \
          | base64 --decode
      )" \
      https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | jq .
    
  2. Does LicenseStatus.State have a value of Config Report Ready to Download?

    • Yes:

      • Your cluster is ready to be licensed (it’s not currently licensed). Go to step 3.

    • No:

  3. Using the secure tunnel, retrieve your cluster’s digital asset ID and store it in the CNE_ASSET_ID environment variable:

    export CNE_ASSET_ID=$(
      curl -s \
          --cacert f5-cwc-api-root.crt \
          --cert f5-cwc-api-client.crt \
          --key f5-cwc-api-client.key \
          --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \
          -H "Authorization: Bearer $(
            kubectl get secret cwc-auth-token -n f5-cne-core \
              -o jsonpath='{.data.token}' \
              | base64 --decode
          )" \
          https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | \
      jq -r .InitialRegistrationStatus.LicenseDetails.DigitalAssetID
    )
    
  4. Using the secure tunnel, retrieve your cluster’s configuration report and store it in the CNE_CFG_REPORT environment variable:

    export CNE_CFG_REPORT=$(
      curl -s \
          --cacert f5-cwc-api-root.crt \
          --cert f5-cwc-api-client.crt \
          --key f5-cwc-api-client.key \
          --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \
          -H "Authorization: Bearer $(
            kubectl get secret cwc-auth-token -n f5-cne-core \
              -o jsonpath='{.data.token}' \
              | base64 --decode
          )" \
          https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/report
    )
    
  5. Send your cluster’s digital asset ID, license key, and configuration report to the F5 licensing server, making sure to replace <licenseKey> with the license key (JSON Web Token, or JWT) you downloaded from MyF5:

    curl -s -X POST https://product.apis.f5.com/ee/v1/entitlements/telemetry \
      -H "Content-Type: application/json" \
      -H "F5-DigitalAssetId: $CNE_ASSET_ID" \
      -H "User-Agent: SPK" \
      -H "Authorization: Bearer <licenseKey>" \
      -d "$CNE_CFG_REPORT" \
      -o license-manifest.json
    

    The F5 licensing server provides you with a file named license-manifest.json, which contains your cluster’s license manifest.

  6. Using the secure tunnel, acknowledge receipt of your cluster’s license manifest:

    Note

    Do not run this command more than once. It can only succeed the first time.

    curl -s -X POST -w "%{http_code}\n" -o /dev/null \
      --cacert f5-cwc-api-root.crt \
      --cert f5-cwc-api-client.crt \
      --key f5-cwc-api-client.key \
      --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \
      -H "Authorization: Bearer $(
        kubectl get secret cwc-auth-token -n f5-cne-core \
          -o jsonpath='{.data.token}' \
          | base64 --decode
      )" \
      https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/receipt \
      -d $(cat license-manifest.json | jq -r .manifest) | jq .
    
  7. If the output of the previous command is not 200, contact F5 Support.

Validate that your cluster is licensed

  1. Using the secure tunnel, retrieve your cluster’s license information using the F5 CWC API:

    curl -s \
      --cacert f5-cwc-api-root.crt \
      --cert f5-cwc-api-client.crt \
      --key f5-cwc-api-client.key \
      --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \
      -H "Authorization: Bearer $(
        kubectl get secret cwc-auth-token -n f5-cne-core \
          -o jsonpath='{.data.token}' \
          | base64 --decode
      )" \
      https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | jq .
    

    Example output

    {
      "Status": {
        "ClusterDetails": {
          "Name": "SPK Cluster"
        },
        "LicenseDetails": {
          "DigitalAssetID": "f06ae970-5b16-4fc1-894e-ce419b928cc9",
          "EntitlementType": "paid",
          "LicenseExpiryDate": "2024-03-05T00:01:13Z",
          "LicenseExpiryInDays": "362"
        },
        "LicenseStatus": {
          "State": "Verification Complete"
        }
      },
      "TelemetryStatus": {
        "NextReport": {
          "StartDate": "2023-03-08 00:03:20.585513148 UTC m=+3093.980461514",
          "EndDate": "2023-03-31 00:03:20 UTC",
          "State": "Telemetry In Progress"
        }
      }
    }
    
  2. If the value of .Status.LicenseStatus.State is not Verification Complete, contact F5 Support.

Terminate the port-forwarding background process

Terminate the process:

kill $CWC_API_PORT_FORWARD_PID

Next steps

Create custom resources to configure the CNE instance as a working BNK (DPU) instance.