Install the F5 CRDs and the CNE core using Helm¶
Introduction¶
The following instructions describe how to perform a clean installation of the F5 custom resource definitions (CRDs) and the CNE core using Helm.
Note
Follow these guidelines:
Before you begin, make sure you’ve met the system requirements and performed all preceding operations in Before you begin.
Install only one CNE core per cluster.
Perform the tasks in order.
Set the release-manifest version¶
Set the
CNE_RELEASE_MANIFEST_VERSIONenvironment variable to the version of the release manifest.Use this table to determine the version of the release manifest associated with a given version of F5 BIG-IP Cloud-Native Edition:
F5 BIG-IP Cloud-Native Edition version
Release-manifest version
2.2.5
2.2.5-3.2226.0-0.0.832
2.2.4
2.2.4-3.2226.0-0.0.722
2.2.3
2.2.3-3.2226.0-0.0.619
2.2.2
2.2.2-3.2226.0-0.0.549
2.2.1
2.2.1-3.2226.0-0.0.511
2.2.0
2.2.0-3.2226.0-0.0.385
Example
$ export CNE_RELEASE_MANIFEST_VERSION=2.2.5-3.2226.0-0.0.832
Set the repository address and log in to the appropriate artifact registry with Helm¶
Are you using a private artifact registry to store the F5 BIG-IP Cloud-Native Edition Helm charts and Docker container images?
Yes:
Set the
CNE_REPOenvironment variable to the address of the F5 BIG-IP Cloud-Native Edition repository in your private artifact registry.Example
$ export CNE_REPO=registry.company.com/f5-big-ip-cne
Log in to your private artifact registry with Helm.
Skip the next task (Log in to F5 Artifact Registry with Helm).
No:
Set the
CNE_REPOenvironment variable to the address of F5 Artifact Registry:$ export CNE_REPO=repo.f5.com
Note
There is only one repository in F5 Artifact Registry, so the address of the registry is also the address of the repository.
Log in to F5 Artifact Registry with Helm¶
Note
Unless you log out (or get logged out), you need to log in to F5 Artifact Registry only once with Helm.
Make sure your computer is connected to the public internet.
In a terminal window on your computer, make sure you’re in the F5 BIG-IP Cloud-Native Edition directory (for example,
f5-big-ip-cne), which contains your F5 Artifact Registry service-account key (cne_pull_64.json).Log in to F5 Artifact Registry with Helm:
$ cat cne_pull_64.json | helm registry login \ --username _json_key_base64 \ --password-stdin repo.f5.com
You can now use Helm commands on your computer to reference Helm charts in F5 Artifact Registry.
Create the namespace for the CNE core and create an image pull secret¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create the namespace for the CNE core:
$ kubectl create namespace f5-cne-core
Did you store Helm charts and Docker container images in a private artifact registry?
Yes:
If your private artifact registry requires authentication, create an image pull secret to allow Kubernetes to pull container images from the registry:
$ kubectl apply -f <privateRegistryPullSecretManifestFilename> \ --namespace f5-cne-core
No:
Create an image pull secret to allow Kubernetes to pull container images from F5 Artifact Registry:
$ kubectl apply -f far-pull-secret.yaml \ --namespace f5-cne-core
Install the F5 CRDs¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Install the F5 common custom resource definitions (CRDs):
$ helm install f5-spk-crds-common \ oci://$CNE_REPO/charts/f5-spk-crds-common \ --version $(./get-cne-chart-version.sh f5-spk-crds-common) \ --namespace f5-cne-core \ --wait
Do you plan to install a BIG-IP Next for Kubernetes (BNK) instance in this cluster?
Yes:
Install the F5 BNK CRDs:
$ helm install f5-spk-crds-service-proxy \ oci://$CNE_REPO/charts/f5-spk-crds-service-proxy \ --version $(./get-cne-chart-version.sh f5-spk-crds-service-proxy) \ --namespace f5-cne-core \ --wait
No:
Install the F5 CNF CRDs:
$ helm install f5-cnf-crds-n6lan \ oci://$CNE_REPO/charts/f5-cnf-crds-n6lan \ --version $(./get-cne-chart-version.sh f5-cnf-crds-n6lan) \ --namespace f5-cne-core \ --wait
Create an override values file for F5 cert-manager¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for F5 cert-manager:
$ touch f5-cert-manager-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
global: imagePullSecrets: - name: far-pull-secret webhook: image: repository: repo.f5.com/images startupapicheck: image: repository: repo.f5.com/images image: repository: repo.f5.com/images cainjector: image: repository: repo.f5.com/images init_container: image: repository: repo.f5.com/images serviceAccount: create: true name: f5-cert-manager logging_sidecar: enabled: true image: repository: repo.f5.com/images fluentd: host: f5-toda-fluentd.f5-cne-core.svc.cluster.local.
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imagePullSecrets: - name: far-pull-secret
with:
imagePullSecrets: []
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install F5 cert-manager¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to thef5-cert-managerservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount f5-cert-manager
Install F5 cert-manager:
$ helm install f5-cert-manager \ oci://$CNE_REPO/charts/f5-cert-manager \ --version $(./get-cne-chart-version.sh f5-cert-manager) \ --namespace f5-cne-core \ --values f5-cert-manager-override-values.yaml \ --wait
Create an override values file for F5 CRD Conversion¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for F5 CRD Conversion:
$ touch f5-crdconversion-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
global: imagePullSecrets: - name: far-pull-secret crdconversion: image: repository: repo.f5.com/images fluentbit_sidecar: image: repository: repo.f5.com/images serviceAccount: create: true name: f5-crdconversion
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imagePullSecrets: - name: far-pull-secret
with:
imagePullSecrets: []
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install F5 CRD Conversion¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to thef5-crdconversionservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount f5-crdconversion
Install F5 CRD Conversion:
$ helm install f5-crdconversion \ oci://$CNE_REPO/charts/f5-crdconversion \ --version $(./get-cne-chart-version.sh f5-crdconversion) \ --namespace f5-cne-core \ --values f5-crdconversion-override-values.yaml \ --wait
Create an override values file for F5 TODA Fluentd¶
Note
F5 TODA Fluentd is one of several components in the F5 Telemetry, Observability, Debugging, and Analytics (TODA) software.
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for F5 TODA Fluentd:
$ touch f5-toda-fluentd-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
imageCredentials: name: far-pull-secret image: repository: repo.f5.com/images persistence: enabled: true storageClass: <storageClassName> f5ingress_logs: enabled: true stdout: true dssm_logs: enabled: true stdout: true dssm_sentinel_logs: enabled: true stdout: true cm_logs: enabled: true stdout: true serviceAccount: create: true name: f5-toda-fluentd tls: enabled: true
Replace
<storageClassName>with the name of one of your storage classes.Note
The storage class you specify must define a type of storage that supports simultaneous mounting by multiple workloads across different nodes.
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imageCredentials: name: far-pull-secret
with:
imageCredentials: name: ""
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install F5 TODA Fluentd¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to thef5-toda-fluentdservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount f5-toda-fluentd
Install F5 TODA Fluentd:
$ helm install f5-toda-fluentd \ oci://$CNE_REPO/charts/f5-toda-fluentd \ --version $(./get-cne-chart-version.sh f5-toda-fluentd) \ --namespace f5-cne-core \ --values f5-toda-fluentd-override-values.yaml \ --wait
Create an override values file for F5 dSSM¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for F5 Distributed Session State Management (dSSM):
$ touch f5-dssm-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
imagePullSecrets: - name: far-pull-secret dssmUpgrader: image: repository: repo.f5.com/images image: repository: repo.f5.com/images cert_client_sidecar: image: repository: repo.f5.com/images sentinel: fluentbit_sidecar: image: repository: repo.f5.com/images fluentd: host: f5-toda-fluentd.f5-cne-core.svc.cluster.local. db: fluentbit_sidecar: image: repository: repo.f5.com/images fluentd: host: f5-toda-fluentd.f5-cne-core.svc.cluster.local. serviceAccount: create: true name: f5-dssm
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imagePullSecrets: - name: far-pull-secret
with:
imagePullSecrets: []
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install F5 dSSM¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to thef5-dssmservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount f5-dssm
Install F5 dSSM:
$ helm install f5-dssm \ oci://$CNE_REPO/charts/f5-dssm \ --version $(./get-cne-chart-version.sh f5-dssm) \ --namespace f5-cne-core \ --values f5-dssm-override-values.yaml \ --timeout 7m0s \ --wait
Note
Because the pods in the F5 dSSM stateful-set workloads are installed serially, the default Helm timeout of 5 minutes is often not enough.
Create an override values file for RabbitMQ¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for RabbitMQ:
$ touch rabbitmq-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
imageCredentials: name: far-pull-secret image: repository: repo.f5.com/images fluentbit_sidecar: image: repository: repo.f5.com/images serviceAccount: create: true name: rabbitmq
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imageCredentials: name: far-pull-secret
with:
imageCredentials: name: ""
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install RabbitMQ¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to therabbitmqservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount rabbitmq
Install RabbitMQ:
$ helm install rabbitmq \ oci://$CNE_REPO/charts/rabbitmq \ --version $(./get-cne-chart-version.sh rabbitmq) \ --namespace f5-cne-core \ --values rabbitmq-override-values.yaml \ --wait
Download and extract the archive file for the f5-cert-gen chart¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Set the
CNE_CHARTenvironment variable tof5-cert-gen.Example
$ export CNE_CHART=f5-cert-gen
Download the archive file for the f5-cert-gen chart:
$ helm pull oci://$CNE_REPO/utils/$CNE_CHART \ --version $(./get-cne-chart-version.sh utils/$CNE_CHART)
Extract the archive file:
$ tar zxvf ${CNE_CHART}-$(./get-cne-chart-version.sh utils/$CNE_CHART).tgz
Extracting the archive file creates a directory named
cert-genin the release-manifest directory.
Create the TLS credentials required to authenticate with the F5 CWC API¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create two manifests that describe secrets for use with the F5 Cluster-Wide Controller (CWC) API—one for the client and one for the server (CWC)—each of which contains a private key, the corresponding certificate, and the signing certificate:
$ sh cert-gen/gen_cert.sh \ -s=api-server \ -a=f5-spk-cwc.f5-cne-core.svc.cluster.local \ -n=1
The script creates the following manifests in the release-manifest directory:
cwc-license-certs.yamlcwc-license-client-certs.yaml
Create the secrets:
$ kubectl create \ -f cwc-license-certs.yaml \ -f cwc-license-client-certs.yaml \ --namespace f5-cne-core
The command creates the following secrets in the
f5-cne-corenamespace:cwc-license-certscwc-license-client-certs
Install the cpcl-key-cm config map¶
Note
CPCL stands for Common Product Component and Libraries, which is the F5 Cluster-Wide Controller (CWC) module that interacts with the F5 licensing system. The cpcl-key-cm config map contains a JSON Web Key Set (JWKS), which is a set of public keys tied to the F5 licensing server that F5 CWC uses when validating information assets delivered by the F5 licensing server.
Install the
cpcl-key-cmconfig map:$ curl -s https://product.apis.f5.com/ee/v1/keys/jwks | \ kubectl create cm cpcl-key-cm \ --from-file=jwt.key=/dev/stdin \ --namespace f5-cne-core
Decide which licensing mode to use¶
Use this table to help you decide which licensing mode is better for you:
Licensing mode |
Description |
|---|---|
Connected |
F5 Cluster-Wide Controller (CWC) connects directly to the F5 licensing server via the public internet. |
Disconnected |
Your cluster is not connected to the public internet. You must manually perform licensing for your cluster. |
Create an override values file for F5 CWC¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Create a file to use as the override values file for F5 Cluster-Wide Controller (CWC):
$ touch f5-cwc-override-values.yaml
Open the file in a text editor.
Add the following information to the file:
imageCredentials: name: far-pull-secret cpclConfig: operationMode: connected jwt: "<licenseKey>" teemCertUrl: "https://product.apis.f5.com/ee/v1" teemEntitlementUrl: "https://product-s.apis.f5.com/ee/v1" teemInitialConfigUrl: "https://product-s.apis.f5.com/ee/v1" friendlyName: "CNE Cluster" rabbitmqNamespace: "f5-cne-core" cert-orchestrator: enabled: true fluentbit_sidecar: enabled: true image: repository: repo.f5.com/images cwc: image: repository: repo.f5.com/images orch: enabled: true image: repository: repo.f5.com/images persistence: enabled: true storageClass: <storageClassName> serviceAccount: create: true name: f5-cwc
Replace
<licenseKey>with the license key (a JSON Web Token, or JWT) supplied to you by F5.Note
Make sure the license key is enclosed in quotation marks.
Replace
<storageClassName>with the name of one of your storage classes.Note
The storage class you specify must define a type of storage that supports simultaneous mounting by multiple workloads across different nodes.
If you want to use disconnected mode instead of connected mode as your licensing mode, change the value of
.cpclConfig.operationModetodisconnected.For help deciding which mode to use, see Decide which licensing mode to use.
(Private artifact registry only) Does your private artifact registry require authentication?
Yes:
Replace all instances of
far-pull-secretwith the name of the pull secret for your private artifact registry (for example,private-registry-pull-secret).
No:
Replace all instances of the following:
imageCredentials: name: far-pull-secret
with:
imageCredentials: name: ""
(Private artifact registry only) Replace all instances of
repo.f5.com/imageswith the location of the Docker container images in your private artifact registry (for example,registry.company.com/f5-big-ip-cne/images).Save and close the file.
Install F5 CWC¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
(OpenShift only) Assign the
privilegedsecurity context constraint (SCC) to thef5-cwcservice account, which will be created by Helm:$ oc adm policy add-scc-to-user privileged \ --namespace f5-cne-core \ --serviceaccount f5-cwc
Install F5 CWC:
$ helm install f5-cwc oci://$CNE_REPO/charts/cwc \ --version $(./get-cne-chart-version.sh cwc) \ --namespace f5-cne-core \ --values f5-cwc-override-values.yaml \ --wait
Prepare to communicate with the F5 CWC API¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Extract the F5 Cluster-Wide Controller (CWC) API client key from the
cwc-license-client-certssecret and store it inf5-cwc-api-client.key:$ kubectl get secret cwc-license-client-certs \ --namespace f5-cne-core -o json | \ jq -r '.data["client-key"]' | \ base64 -d > f5-cwc-api-client.key
Extract the F5 CWC API client certificate from the
cwc-license-client-certssecret and store it inf5-cwc-api-client.crt:$ kubectl get secret cwc-license-client-certs \ --namespace f5-cne-core -o json | \ jq -r '.data["client-cert"]' | \ base64 -d > f5-cwc-api-client.crt
Extract the F5 CWC API root certificate from the
cwc-license-client-certssecret and store it inf5-cwc-api-root.crt:$ kubectl get secret cwc-license-client-certs \ --namespace f5-cne-core -o json | \ jq -r '.data["ca-root-cert"]' | \ base64 -d > f5-cwc-api-root.crt
Use port forwarding to create a secure tunnel between your computer and the F5 CWC API:
$ kubectl port-forward service/f5-spk-cwc 38081 \ --namespace f5-cne-core > /dev/null & disown; \ CWC_API_PORT_FORWARD_PID=$!
Note
This command:
Forwards port 38081 on your computer to the target port associated with port 38081 of a pod in the
f5-spk-cwcservice in thef5-cne-corenamespace (the F5 CWC API)Suppresses normal output (
> /dev/null)Runs the process in the background (
&)Detaches the process from the shell to avoid creating an associated job (
disown;)Saves the process ID so you can later terminate the process
(Disconnected mode only) License your cluster manually¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Using the secure tunnel, retrieve your cluster’s license information using the F5 CWC API:
$ curl -s \ --cacert f5-cwc-api-root.crt \ --cert f5-cwc-api-client.crt \ --key f5-cwc-api-client.key \ --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \ https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | jq .
Does
LicenseStatus.Statehave a value ofConfig Report Ready to Download?Yes:
Your cluster is ready to be licensed (it’s not currently licensed). Go to step 4.
No:
Does
.Status.LicenseStatus.Statehave a value ofVerification Complete?Yes:
Your cluster is already licensed. Go to Terminate the port-forwarding background process.
No:
Contact F5 support.
Using the secure tunnel, retrieve your cluster’s digital asset ID and store it in the
CNE_ASSET_IDenvironment variable:$ export CNE_ASSET_ID=$( curl -s \ --cacert f5-cwc-api-root.crt \ --cert f5-cwc-api-client.crt \ --key f5-cwc-api-client.key \ --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \ https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | \ jq -r .InitialRegistrationStatus.LicenseDetails.DigitalAssetID )
Using the secure tunnel, retrieve your cluster’s configuration report and store it in the
CNE_CFG_REPORTenvironment variable:$ export CNE_CFG_REPORT=$( curl -s \ --cacert f5-cwc-api-root.crt \ --cert f5-cwc-api-client.crt \ --key f5-cwc-api-client.key \ --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \ https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/report )
Send your cluster’s digital asset ID, license key, and configuration report to the F5 licensing server, making sure to replace
<licenseKey>with the license key (a JSON Web Token, or JWT) supplied to you by F5:$ curl -s -X POST https://product.apis.f5.com/ee/v1/entitlements/telemetry \ -H "Content-Type: application/json" \ -H "F5-DigitalAssetId: $CNE_ASSET_ID" \ -H "User-Agent: SPK" \ -H "Authorization: Bearer <licenseKey>" \ -d "$CNE_CFG_REPORT" \ -o license-manifest.json
The F5 licensing server provides you with a file named
license-manifest.json, which contains your cluster’s license manifest.Using the secure tunnel, acknowledge receipt of your cluster’s license manifest:
Note
Do not run this command more than once. It can only succeed the first time.
$ curl -s -X POST -w "%{http_code}\n" -o /dev/null \ --cacert f5-cwc-api-root.crt \ --cert f5-cwc-api-client.crt \ --key f5-cwc-api-client.key \ --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \ https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/receipt \ -d $(cat license-manifest.json | jq -r .manifest) | jq .
If the output of the previous command is not
200, contact F5 Support.
Validate that your cluster is licensed¶
In a terminal window on your computer, make sure you’re in the release-manifest directory.
Using the secure tunnel, retrieve your cluster’s license information using the F5 CWC API:
curl -s \ --cacert f5-cwc-api-root.crt \ --cert f5-cwc-api-client.crt \ --key f5-cwc-api-client.key \ --connect-to f5-spk-cwc.f5-cne-core.svc.cluster.local:38081:localhost:38081 \ https://f5-spk-cwc.f5-cne-core.svc.cluster.local:38081/status | jq .
Example output
{ "Status": { "ClusterDetails": { "Name": "SPK Cluster" }, "LicenseDetails": { "DigitalAssetID": "f06ae970-5b16-4fc1-894e-ce419b928cc9", "EntitlementType": "paid", "LicenseExpiryDate": "2024-03-05T00:01:13Z", "LicenseExpiryInDays": "362" }, "LicenseStatus": { "State": "Verification Complete" } }, "TelemetryStatus": { "NextReport": { "StartDate": "2023-03-08 00:03:20.585513148 UTC m=+3093.980461514", "EndDate": "2023-03-31 00:03:20 UTC", "State": "Telemetry In Progress" } } }
If the value of
.Status.LicenseStatus.Stateis notVerification Complete, contact F5 Support.
Terminate the port-forwarding background process¶
Terminate the process:
$ kill $CWC_API_PORT_FORWARD_PID
The F5 custom resource definitions (CRDs) and the CNE core are now installed.