SPK CWC¶
Overview¶
The Service Proxy for Kubernetes (SPK) Cluster Wide Controller (CWC) enables SPK’s software licensing and billing capabilities. Once the SPK software is installed and licensed, the CWC collects and reports software usage telemetry statistics based on the number of SPK CRs used by the licensed BIG-IP Controller instances in the cluster. SPK uses F5’s flexible consumption software licensing model, billing only for the SPK features used.
Note: SPK Licensing applies to the cluster level, and is performed prior to installing the BIG-IP Controller instances.
This document guides you through installing the CWC controller.
CPCL module¶
The CWC contains the Common Product Component and Libraries (CPCL) module that helps with license activation, and with generating and maintaining the monthly license reports. The CPCL requires these two important licensing objects:
A JWKS (JSON Web Key Set) that must be installed prior to installing the CWC Pod. The JWKS can be downloaded and installed using the Install the JWKS procedure.
A JWT (JSON Web Token) that is associated with your unique CAT (Customer Association Token). The JWT can be obtained from your MyF5 account and will be installed using the Install the CWC procedure.
Installing the CWC Pod is demonstrated in this overview, and licensing the cluster will demonstrated in the SPK Licensing overview.
CPCL modes¶
The CPCL module supports three licensing modes:
disconnected - User can download the report and upload to the F5 TEEM system.
connected - User can directly connect the BNK, SPK, or CNE cluster instances to the F5 TEEM system.
F5 License Proxy (FLP) - User can use F5 License Proxy to connect BNK, SPK, or CNE cluster instances (configured in FLP mode) with the F5TEEM system.
The CWC configurations required to enable each mode are available in the Procedures section of this document.
Cluster namespace¶
The CWC Pod can install to any cluster namespace. In this document, the CWC will install to the spk-telemetry namespace. As mentioned previously, CWC licensing applies to the entire cluster, not a single namespace (namespace).
RabbitMQ¶
The CWC and BIG-IP Controller Pod communicate through the RabbitMQ open source message broker to determine the cluster licensing status. Ensure connectivity is permitted for the service ports listed in the sections below.
CWC Service¶
After installing the CWC, a CWC Service object is created that receives REST API data on TCP service port 30881, and forwards the data to the CWC Pod on TCP service port 38081. Ensure the Service ports are available, and the cluster has CoreDNS enabled. In this example, the SPK components will need to resolve the f5-spk-cwc.spk-telemetry DNS hostname.
Name: f5-spk-cwc
Namespace: spk-telemetry
IP: 10.109.102.215
Port: cwc-rest 30881/TCP
Endpoints: 10.244.1.75:38081
RabbitMQ Service¶
After installing the RabbitMQ Pod, a RabbitMQ Service object is created, to pass messages between the BIG-IP Controllers and the CWC on TCP service port 5671. Ensure the Service port is available, and the cluster’s core DNS is enabled. In this example, the SPK components will need to resolve the rabbitmq-server.spk-telemetry hostname.
Name: rabbitmq-server
Namespace: spk-telemetry
IP: 10.109.105.210
Port: ampqst 5671/TCP
Endpoints: 10.244.1.80:5671
CWC capabilities¶
The CWC supports the following capabilities:
Licensing. Refer to the SPK Licensing guide for more information.
Debugging. Refer to the Debug API guide for more information.
Qkview. Refer to the Qkview API guide for more information.
Requirements¶
Ensure you have:
Installed the SPK software.
Installed the SPK Cert Manager.
Obtained the CPCL SSL/TLS key and the JWT from your MyF5 account.
Procedures¶
Create API certificates¶
Use this procedure to create the SSL/TLS certificates required to authenticate the CWC REST API when using CWC in disconnected mode.
Change into directory with the SPK Software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory.
cd spkinstall
ls -1 tar
This procedure requires the f5-cert-gen-0.9.3.tgz file.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
Extract the cert-gen utility to generate the SSL/TLS certificates and Secrets:
tar xvf tar/f5-cert-gen-0.9.3.tgz
Generate the SSL/TLS certificates and Secret for the CWC REST API:
Note: The certificates will be referenced in the Configure Postman section of the SPK Licensing guide.sh cert-gen/gen_cert.sh -s=api-server -a=f5-spk-cwc.<namespace> -n=1
In this example, the CWC installs to the spk-telemetry namespace.
sh cert-gen/gen_cert.sh -s=api-server -a=f5-spk-cwc.spk-telemetry -n=1
The command output indicates the Secret has been created:
Generating /path/cwc-license-certs.yamlInstall the CWC REST API Secret:
In this example, the CWC installs to the spk-telemetry Project.
oc apply -f cwc-license-certs.yaml -n spk-telemetry
The command output indicates the Secret was created successfully:
secret/cwc-license-certs createdContinue to the next procedure.
Install RabbitMQ¶
Use these steps to install the RabbitMQ Pod.
Change into directory with the SPK Software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory.
cd spkinstall
ls -1 tar
This procedure requires the rabbitmq-0.6.1-0.0.13.tgz file.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
To pull the RabbitMQ software image from the local software registry, create a rabbitmq-values.yaml file, and set the
image.repositoryparameter:rabbitmq: image: repository: "local.registry.com" fluentbit_sidecar: image: repository: "local.registry.com"
Install the RabbitMQ Pod:
In this example, the RabbitMQ Pod installs to the spk-telemetry namespace.
helm install spk-rabbit tar/rabbitmq-0.6.1-0.0.13.tgz -f rabbitmq-values.yaml -n spk-telemetry
Verify the RabbitMQ Pod STATUS:
oc get pods -n spk-telemetry
In this example, the RabbitMQ Pod STATUS is Running.
NAME READY STATUS f5-rabbit-5688f9c8c7-f7d9d 1/1 Running
Continue to the next section.
Install the JWKS¶
Use these steps to download and install the JWKS ConfigMap.
Download the JWKS cpcl-key-cm (key) ConfigMap here.
Install the JWKS cpcl-key-cm ConfigMap:
In this example, the ConfigMap installs to the spk-telemetry namespace:
oc apply -f cpcl-key.yaml -n spk-telemetry
Continue to either the Install CWC - connected or the Install CWC - F5 License Proxy (FLP) or the Install CWC - disconnected procedure.
Install CWC - connected¶
Use these steps to install the CWC Pod using connected mode.
Important: Ensure the CWC Pod has access to the product.apis.f5.com licensing server over TCP service port 443.
Change into the directory with the SPK software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory:
cd spkinstall
ls -1 tar
This procedure requires the cwc-0.43.1-0.0.15.tgz Helm chart.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
To pull the CWC software image from the local software registry, create a cwc-values.yaml file, and set the
image.repositoryparameter:cwc: image: repository: "local.registry.com" name: spk-cwc orch: image: repository: "local.registry.com" name: f5-csm-qkview fluentbit_sidecar: image: repository: "local.registry.com" name: fluentbit
Set the
rabbitmqNamespaceparameter to the CWC namespace in the cwc-values.yaml file:rabbitmqNamespace: "spk-telemetry"
To automatically obtain the cluster license from the F5 license server, add the F5 provided JWT to the
jwtparameter in the cwc-values.yaml file:Note: The
jwtvalue “eyJhbGciOiJSUzUxMiIsInR5cCI6” has been shortened for readability.cpclConfig: operationMode: "connected" jwt: "eyJhbGciOiJSUzUxMiIsInR5cCI6" teemCertUrl: "https://product.apis.f5.com/ee/v1/entitlements/telemetry" teemEntitlementUrl: "https://product.apis.f5.com/ee/v1/entitlements/telemetry" teemInitialConfigUrl: "https://product.apis.f5.com/ee/v1/entitlements/telemetry"
Add the SPK CWC default serviceAccount to the Project’s privileged security context constraint (SCC):
oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
In this example, the default serviceAccount is added to the spk-telemetry Project’s privileged SCC:
oc adm policy add-scc-to-user privileged -n spk-telemetry -z default
Install the CWC Pod:
In this example, the CWC Pod installs to the spk-telemetry namespace.
helm install spk-cwc tar/cwc-0.43.1-0.0.15.tgz -f cwc-values.yaml -n spk-telemetry
Verify the STATUS of the CWC Pod:
oc get pods -n spk-telemetry
In this example, the CWC Pod STATUS is Running.
NAME READY STATUS f5-rabbit-5688f9c8c7-lv49b 1/1 Running f5-spk-cwc-94bcd64bd-42xdc 1/1 Running
Continue to the Next steps section.
Install CWC - F5 License Proxy (FLP)¶
Use these steps to install the CWC Pod using FLP mode.
Important: Ensure the CWC Pod has access to the F5-License-proxy (FLP_HOSTNAME) over TCP service port 30001
Install and configure F5 License Proxy (FLP), see Install and configure F5 License Proxy (FLP).
The SPK cluster needs FLP RootCA to trust server cert of F5 License Proxy. Perform the following command to get the FLP root CA on the FLP VM and save the cert in a file (For eg: certificate.crt)
setup --print-ca
Sample Output
Secret 'flp-mtls-secret' exists. CA certificate found, decoding from Base64... -----BEGIN CERTIFICATE----- MIICLzCCAdWgAwIBAgIUA48LLaJoJeOQ9WGYMqGRY2CGfkcwCgYIKoZIzj0EAwIw bTELMAkGA1UEBhMCVVMxEzARBgNVBAgMCldhc2hpbmd0b24xEDAOBgNVBAcMB1Nl YXR0bGAgEGCCqGSM49AwEHA0IABFWNOAU5ieEIasdsfNUfnkRajgsdySjyhmPnbat -----END CERTIFICATE-----
Change into the directory with the SPK software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory:
cd spkinstall
ls -1 tar
This procedure requires the cwc-0.43.1-0.0.15.tgz Helm chart.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
To pull the CWC software image from the local software registry, create a cwc-values.yaml file, and set the
image.repositoryparameter:cwc: image: repository: "local.registry.com" name: spk-cwc orch: image: repository: "local.registry.com" name: f5-csm-qkview fluentbit_sidecar: image: repository: "local.registry.com" name: fluentbit
Set the
rabbitmqNamespaceparameter to the CWC namespace in the cwc-values.yaml file:rabbitmqNamespace: "spk-telemetry"
To automatically obtain the cluster license from the F5 license server, add the F5 provided JWT to the
jwtparameter in the cwc-values.yaml file:Note: The
jwtvalue “eyJhbGciOiJSUzUxMiIsInR5cCI6” has been shortened for readability.cpclConfig: operationMode: "f5licenseproxy" jwt: "eyJhbGciOiJSUzUxMiIsInR5cCI6" teemCertUrl: "https://${FLP_HOSTNAME}:30001/license-proxy/v1" teemEntitlementUrl: "https://${FLP_HOSTNAME}:30001/license-proxy/v1" teemInitialConfigUrl: "https://${FLP_HOSTNAME}:30001/license-proxy/v1"
Add the SPK CWC default serviceAccount to the Project’s privileged security context constraint (SCC):
oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
In this example, the default serviceAccount is added to the spk-telemetry Project’s privileged SCC:
oc adm policy add-scc-to-user privileged -n spk-telemetry -z default
Provide the following values of FLP to install SPK CWC using cwc.yaml.
cpclConfig.operationMode=”f5licenseproxy”
cpclConfig.teemCertUrl=”https://${FLP_HOSTNAME}:30001/license-proxy/v1”
cpclConfig.teemEntitlementUrl=”https://${FLP_HOSTNAME}:30001/license-proxy/v1”
cpclConfig.teemInitialConfigUrl=”https://${FLP_HOSTNAME}:30001/license-proxy/v1”
licenseserverrootca.crt=certificate.crt
You can use the following helm command for installing SPK CWC and helm values.
helm install cwc https://artifactory.f5net.com/artifactory/f5-f5ingress-helm-local/f5-spk-cwc-${CWC_VERSION}.tgz -f cwc-values.yaml
--set cpclConfig.operationMode="f5licenseproxy"
--set cpclConfig.teemCertUrl="https://${FLP_HOSTNAME}:30001/license-proxy/v1"
--set cpclConfig.teemEntitlementUrl="https://${FLP_HOSTNAME}:30001/license-proxy/v1"
--set cpclConfig.teemInitialConfigUrl="https://${FLP_HOSTNAME}:30001/license-proxy/v1"
--set-file licenseserverrootca.crt=certificate.crt
Verify the STATUS of the CWC Pod:
oc get pods -n spk-telemetry
In this example, the CWC Pod STATUS is Running.
NAME READY STATUS f5-rabbit-5688f9c8c7-lv49b 1/1 Running f5-spk-cwc-94bcd64bd-42xdc 1/1 Running
Continue to the Next steps section.
Install CWC - disconnected¶
Use these steps to install the CWC Pod using disconnected mode.
Change into the directory with the SPK software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory:
cd spkinstall
ls -1 tar
This procedure requires the cwc-0.43.1-0.0.15.tgz Helm chart.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
To pull the CWC software image from the local software registry, create a cwc-values.yaml file, and set the
image.repositoryparameter:cwc: image: repository: "local.registry.com" name: spk-cwc orch: image: repository: "local.registry.com" name: f5-csm-qkview fluentbit_sidecar: image: repository: "local.registry.com" name: fluentbit
Set the
rabbitmqNamespaceparameter to the CWC namespace in the cwc-values.yaml file:rabbitmqNamespace: "spk-telemetry"
Add the SPK CWC serviceAccount to the Project’s privileged security context constraint (SCC):
Note: The spk-cwc serviceAccount name is based on the Helm release name. See Step 6.oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
In this example, the spk-cwc serviceAccount is added to the spk-telemetry Project’s privileged SCC:
oc adm policy add-scc-to-user privileged -n spk-telemetry -z spk-cwc
Install the CWC Pod, and reference the JWT:
In this example, the
cpcpConfig.jwtvalue eyJhbGciOiJSUzUxMiIsInR5cCI6 has been truncated for readabilityhelm install spk-cwc tar/cwc-0.43.1-0.0.15.tgz -f cwc-values.yaml \ --set cpclConfig.jwt=eyJhbGciOiJSUzUxMiIsInR5cCI6 -n spk-telemetry
Verify the STATUS of the CWC Pod:
oc get pods -n spk-telemetry
In this example, the CWC Pod STATUS is Running.
NAME READY STATUS f5-rabbit-5688f9c8c7-lv49b 1/1 Running f5-spk-cwc-94bcd64bd-42xdc 1/1 Running
Continue to the Next steps section.
Create Qkview API certificates¶
Use this procedure to create the SSL/TLS certificates required to authenticate the CWC Qkview REST API when using CWC in disconnected mode.
Change into directory with the SPK Software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory.
cd spkinstall
ls -1 tar
This procedure requires the f5-cert-gen-0.9.3.tgz file.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
Extract the cert-gen utility to generate the SSL/TLS certificates and Secrets:
tar xvf tar/f5-cert-gen-0.9.3.tgz
Generate the SSL/TLS certificates and Secret for the CWC Qkview REST API:
Note: The certificates will be referenced in the Configure Postman section of the SPK Licensing guide.sh cert-gen/gen_cert.sh -s=qkview -a=f5-spk-cwc.<namespace> -n=1
In this example, the CWC installs to the spk-telemetry namespace.
sh cert-gen/gen_cert.sh -s=qkview -a=f5-spk-cwc.spk-telemetry -n=1
The command output indicates the Secret has been created:
Generating /path/qkview-server-certs.yaml /path/qkview-client-certs.yaml
Install the CWC Qkview REST API Secret: (client-cert)
In this example, the CWC installs to the spk-telemetry Project.
oc apply -f qkview-client-certs.yaml -n spk-telemetry
The command output indicates the Secret was created successfully:
secret/qkview-client-certs createdInstall the CWC Qkview REST API Secret: (server-cert)
In this example, the CWC installs to the spk-telemetry Project.
oc apply -f qkview-server-certs.yaml -n <namespace_where_qkview_pods_are_running>
The command output indicates the Secret was created successfully:
secret/qkview-server-certs createdContinue to the next procedure.
Install csm-orchestrator within cwc to run qkview process¶
Use these steps to install the CWC Pod with csm-orchestrator container to run qkview process.
Change into the directory with the SPK software files, and list the files in the tar directory:
In this example, the SPK files are in the spkinstall directory:
cd spkinstall
ls -1 tar
This procedure requires the cwc-0.43.1-0.0.15.tgz Helm chart.
csrc-0.9.1-0.3.0.tgz cwc-0.43.1-0.0.15.tgz coremond-0.7.56-0.0.5.tgz dnat-util-v0.5.6.tgz f5-cert-gen-0.9.3.tgz f5-cert-manager-0.23.35-0.0.10.tgz f5-crdconversion-0.23.2-0.1.1.tgz f5-dssm-1.27.1-0.0.20.tgz f5-toda-fluentd-1.31.30-0.0.7.tgz f5ingress-v13.7.1-0.3.22.tgz log-doc-f5ingress-13.7.1+0.3.22.tgz rabbitmq-0.6.1-0.0.13.tgz f5-toda-observer-5.7.5-0.1.0.tgz cne-docker-images.tgz
To pull the CWC software image from the local software registry, create a cwc-values.yaml file, and set the
image.repositoryparameter:cwc: image: repository: "local.registry.com" name: spk-cwc orch: image: repository: "local.registry.com" name: f5-csm-qkview fluentbit_sidecar: image: repository: "local.registry.com" name: fluentbit
Set the
rabbitmqNamespaceparameter to the CWC namespace in the cwc-values.yaml file:rabbitmqNamespace: "spk-telemetry"
Add the SPK CWC serviceAccount to the Project’s privileged security context constraint (SCC):
Note: The spk-cwc serviceAccount name is based on the Helm release name. See Step 6.oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
In this example, the spk-cwc serviceAccount is added to the spk-telemetry Project’s privileged SCC:
oc adm policy add-scc-to-user privileged -n spk-telemetry -z spk-cwc
Install the CWC Pod, and reference the JWT:
In this example, the
cpcpConfig.jwtvalue eyJhbGciOiJSUzUxMiIsInR5cCI6 has been truncated for readabilityhelm install spk-cwc tar/cwc-0.43.1-0.0.15.tgz -f cwc-values.yaml \ --set cpclConfig.jwt=eyJhbGciOiJSUzUxMiIsInR5cCI6 -n spk-telemetry \ --set orchContainer.enabled=true --set persistence.enabled=true
Verify the STATUS of the CWC Pod:
oc get pods -n spk-telemetry
In this example, the CWC Pod STATUS is Running.
NAME READY STATUS f5-rabbit-5688f9c8c7-lv49b 1/1 Running f5-spk-cwc-94bcd64bd-42xdc 2/2 Running
Continue to the Next steps section.
Next steps
If the CPCL is running in
connectedmode, continue to the SPK Licensing guide to license the cluster.If the CPCL is running in
disconnectedmode, continue to the SPK Licensing guide to license the cluster.If the CPCL is running in
FLPmode, continue to the SPK Licensing guide to license the cluster.
Feedback
Provide feedback to improve this document by emailing spkdocs@f5.com.
Supplemental