Last updated on: Apr 13, 2023.

SPK Licensing

Licensing

The new subscription based licensing model being introduced for SPK (and indeed for modular BIG-IP) differs markedly from the traditional licensing model used by classical BIG-IP (cBIP).

1. For cBIP, Virtual Machines are licensed individually and the license describes at a module level which features can be used.

2. For SPK, features are not licensed. By default all the features in the TMM are enabled and the customer is billed on their usage metrics, one of which being their consumption of Kubernetes Custom Resources Definitions (CRDs).

The SPK Controller only applies the configuration for infrastructure CRs (eg. SR-IOV, VLANs etc) and use-case CRs upon receiving license verification messages from CWC

Telemetry

Telemetry is key to the success of this new “license without license” model, since without it F5 does not know which features a customer is using and hence cannot bill them correctly for the usage they have incurred. It operates like current public cloud based utility billing models, where metric data must be periodically ingested and used for charging.

Pricing Model

The current pricing model for SPK is based only on the number of Virtual CPUs (vCPUs) in the cluster and the usage report contains only 3 values: cpu_count, unique_crds, total_crds. All of these values can be gathered by CWC using Kubernetes APIs.

In future, the usage of individual CRDs will be added to the telemetry to show the consumption of a given CRD across all namespaces. One idea is to deliver Layer 4 CRDs at a lower price point and higher logic Layer 7 CRDs at a higher price point, similar to the Good, Better, Best pricing model adopted by classic BIG-IP.

Modes of Operation

How F5 receives this telemetry report depends on how the customer wants to setup and use their service. There is currently the option of either Direct Connect or Disconnected, where the latter is the focus here.

1. Direct Connect: Report is directly posted to F5 by the product and a signed acknowledgement is directly received. No intervention is required by the customer.

2. Disconnected: Customer interfaces with F5 via API. Customer then receives a signed acknowledgment file.

NOTE: Disconnected applies when the customer’s environment has to be firewalled (or isolated) from the Internet for security. This currently requires manual intervention, but it should be largely avoidable when a Mediated Connection using a Local License Manager (LLM) is integrated into SPK. LLM automatically proxies communication with F5. The product then reports to LLM and LLM is responsible for providing a signed acknowledgment. The LLM may locally acknowledge and then later send an aggregated report to F5 with its own signed acknowledgement.

Customer Association Token (CAT)

When a user becomes a customer of F5 they will be assigned one or more CATs. A CAT is a long text string that uniquely identifies this user as a customer and allows this user to activate F5 products and establish entitlement. It is product agnostic. If a customer has multiple subscriptions with differing end dates, they will be assigned multiple CATs. The customer will retrieve their CATs from MyF5

Digital Access ID

The Digital Access ID is an immutable 128 bit Universally Unique IDentifier (UUID) that is assigned to the product and which will stay with it for its whole lifecycle. It is used in API calls to uniquely identify the product to the licensing service and appears as the digitalAssetID in the ProductTelemetry object (see later)

The CPCL license plugin creates the digitalAssetID for any product instance.

Common Product Components and Libraries team (CPCL)

The CPCL team has developed a Licensing Service for the NGINX Controller which is now being adopted by SPK. A component of this will be the CPCL License Adapter which will be integrated into MyF5: the new digital support experience from F5 Support. It can be used to track and present a consolidated view of all F5 customer licenses.

Cluster Wide Controller (CWC)

CWC, as its name implies, supports the cluster wide software licensing and billing capabilities of every SPK Controller operating in that SPK cluster, irrespective of namespace. It is a Kubernetes Pod which should be deployed by default in the default name space. There should be only one instance of CWC running in a cluster.

Licensing Policy

The following customer guidelines currently apply to the use of such SPK licenses

1. If the license has not been activated and the cluster is not in production, any licensing errors (e.g. CWC uninstall, CWC unreachable etc.) are deemed fatal and traffic must be blocked.

2. Following receipt of the first telemetry acknowledgement (which activates the license), the production use of that license should not be interrupted. It is irrelevant whether the license has expired, there is a failure in creating, downloading or acknowledging a telemetry report or any other licensing related error has occurred. There will be warning messages but normal operation will be allowed to proceed.

3. If the customer only has a trial license, any errors including license expiry are deemed fatal and traffic must be blocked.

Blocking is implemented when required by instructing the SPK Controllers to scale down the number of available TMM instances to 0.

RabbitMQ

A RabbitMQ open-source message broker which is used to pass messages associated with licensing verification and troubleshooting between the SPK Controllers and the CWC on TCP service port 5671. There are bidirectional message queues between CWC and each SPK Controller. In one queue CWC acts as the producer and the SPK Controller acts as the consumer and in another queue CWC acts as consumer and SPK Controller as the producer. There are also separate message queues for each SPK instance so that one SPK instance cannot block messages destined for another instance.

Until v1.6.1 RabbitMQ is deployed as side-car container named spk-cwc. From v1.7.0, the role of rabbitMQ will be extended to support a variety of micro-services and it will be deployed in its own Pod with its own HELM chart.

Heartbeat

When SPK is initially brought up, the SPK Controller sends a “GetEntitlementRequest” message to CWC. If the license cannot be verified CWC responds with “GetbackLater” which SPK polls every 5 seconds. If the license is already verified, CWC sends “Verified” based on JWT used. Once the SPK Controller receives an “Entitlement Paid” response from CWC, it stops sending “GetEntitlementRequest” and instead, CWC sends a “heartbeat” message to each SPK Controller every 60s.

If communication between the CWC and SPK Controller is lost, the license can no longer be enforced and TMM will be instructed by the SPK Controller (in accordance with current licensing policy) to stop passing traffic.

For resilience, CWC persists its own state so should it fail, it should be restored by Kubernetes to its last known operating state upon boot-up.

CWC stores state, error and context information in Kubernetes secrets to handle the case where a pod is deleted accidentally.

pmills@npi:~/tar/cert-gen$ oc logs deployment/f5-spk-cwc -n default -c spk-cwc | tail
I1123 14:20:34.106219       1 cm20.go:105] ====> Published heartbeat message
I1123 14:21:34.106962       1 cm20.go:440] DEBUG: exchange name = CWC-SPK

Troubleshooting

The UNIX/Linux commands rabbitmq-diagnostics and rabbitmqctl can be used to troubleshoot RabbitMQ. Notice how RabbitMQ supports both licensing and the SPK debug-api. Also notice the different message queues mentioned earlier.

oc exec -it deployment.apps/f5-spk-cwc -n default -c rabbitmq-server -- bash
bash-5.1$ rabbitmqctl list_bindings
Listing bindings for vhost /...
source_name     source_kind     destination_name        destination_kind        routing_key     arguments
CWC-SPK exchange        debug-api/peter-net/f5-tmm-6c96c59856-lgmsw     queue   debug-api/peter-net/f5-tmm-6c96c59856-lgmsw     []
CWC-SPK exchange        debug-api/peter-net/f5-tmm-6cfcb8f678-hs2st     queue   debug-api/peter-net/f5-tmm-6cfcb8f678-hs2st     []
CWC-SPK exchange        debug-api/peter-net/f5-tmm-bdf7f99-h5tkk        queue   debug-api/peter-net/f5-tmm-bdf7f99-h5tkk        []
CWC-SPK exchange        spk-peter-net   queue   heartbeat       []
CWC-SPK exchange        spk-peter-net   queue   license []
CWC-SPK exchange        spk-peter-net   queue   spk-peter-net   []
...
bash-5.1$ rabbitmq-diagnostics list_consumers queue_name active
Listing consumers in vhost / ...
queue_name      active
debug-api/peter-net/f5-tmm-68f7c68776-n8x7c     true
cwc     true
spk-peter-net   true
...
bash-5.1$ rabbitmq-diagnostics log_tail --number 10
Last 10 log lines on node rabbit@f5-spk-cwc-6686c46bd7-8mgv2 ...
2023-02-06 23:43:45.563732+00:00 [info] <0.29798.105> connection <0.29798.105> (10.131.0.139:41402 -> 10.128.3.82:5671): user 'f5net' authenticated and granted access to vhost '/'
2023-02-06 23:58:17.143411+00:00 [warning] <0.29798.105> closing AMQP connection <0.29798.105> (10.131.0.139:41402 -> 10.128.3.82:5671, vhost: '/', user: 'f5net'):
2023-02-06 23:58:17.143411+00:00 [warning] <0.29798.105> client unexpectedly closed TCP connection

CPCL Functionality

The CWC Pod includes a CPCL license plugin to help activate licenses and generate monthly license reports.

It allows the CWC to

1. decode the license blob it holds about SPK in the Kubernetes cluster.

NOTE: There is only one license to entitle all the SPK Controllers running in a cluster.

1. communicate with the upstream CPCL infrastructure.

2. provide an API service which can handle license queries (user case entitlements) made by the SPK Controller.

The CPCL requires a F5 provided SSL/TLS key (JWK) and JSON Web Token (JWT) to identify the cluster.

CWC REST API

This API supports the following endpoints

Method	Endpoint	Description
GET	/status	Check status
GET	/report	Download the initial configuration report applicable only in disconnected mode
POST	/receipt	Upload the signed acknowledgement pending verification by CPCL applicable only in disconnected mode
POST	/reactivate	Upload a new JSON Web Token for a new or updated license, e.g. change from Trial to Paid. A verified and valid license must already be installed on the product. When a license is renewed, the information is only digested by CWC. The SPK Controller need not be informed if everything is still good

Certificates

The following secrets must be configured on a SPK cluster to activate SPK

Google Remote Procedure Calls (gRPC)

Although gRPC is not specific to this particular use-case, it is still a fundamental part of the provisioning process so is mentioned here for completeness

Generate the gRPC secrets required for the SPK Controller (F5Ingress) to talk to tmm,

oc get pods -n peter-net
NAME                                   READY   STATUS    RESTARTS   AGE
f5-tmm-bdf7f99-s8xpm                   2/2     Running   0          22d
f5ingress-f5ingress-7d4b57bd46-mh5hn   2/2     Running   0          22d

by running the following script which will generate the Kubernetes manifests keys-secret.yaml and certs-secret.yaml required for the associated secret objects

oc apply -f keys-secret.yaml -n peter-net
secret/keys-secret created
oc apply -f certs-secret.yaml -n peter-net
secret/certs-secret created

gen_cert.sh

Some of the other secret objects are generated using the script cert-gen/gen_cert.sh supplied in the spk tarball. This script can create multiple client certificates signed by the same Certificate Authority (CA).

tar -xvf f5-spk-tarball-1.6.0.tgz
f5-spk-images-1.6.0.tgz
f5-spk-crds-deprecated-4.0.3.tgz
f5-spk-crds-service-proxy-4.0.3.tgz
f5-spk-crds-common-4.0.3.tgz

tar -xvf f5-spk-images-1.6.0.tgz
tar/csrc-0.1.4.tgz
tar/cwc-0.5.0.tgz
tar/f5-cert-gen-0.2.4.tgz
tar/f5-dssm-0.22.18.tgz
tar/f5-toda-fluentd-1.10.1.tgz
tar/f5ingress-7.0.13.tgz
tar/spk-docker-images.tgz

tar -xvf f5-cert-gen-0.2.4.tgz
cert-gen/
cert-gen/basic/
cert-gen/basic/openssl.cnf
cert-gen/basic/profile.py
cert-gen/basic/.DS_Store
cert-gen/basic/Makefile
cert-gen/basic/CertificateGenerator.md
cert-gen/gen_cert.sh
cert-gen/LICENSE

CPCL

To generate the secrets required for the CPCL REST API Server, where n is the number of client certificates needed, run gen_cert.sh with the following arguments. When n defaults to 0 only one client certificate will be generated.

sh cert-gen/gen_cert.sh -s=api-server -a=f5-spk-cwc.default -n=2

This base64 encodes the secrets ca_root.crt, server.crt, and server.key and adds them (with proper formatting) to the Kubernetes manifest cwc-license-certs.yaml which must then be deployed.

oc apply -f cwc-license-certs.yaml -n default
secret/cwc-license-certs created
oc get secrets -n default cwc-license-certs
cwc-license-certs               Opaque                                3      22d

RabbitMQ

The following log message will be seen until the rabbitMQ certificates have been successfully applied

oc logs deployment/f5ingress-f5ingress -n peter-net -c f5-lic-helper | tail
E1125 13:18:59.918590 1 rabbitmqapi.go:79] Failed to establish connection to rabbitmq server. Error x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "TLSGenSelfSignedtRootCA")
E1125 13:18:59.918630 1 rabbitmq_handler.go:38] Failed to create AMQP. Error Failed to Setup exchange: Failed to open channel

Here Advanced Message Queuing Protocol (AMQP) includes a set of standards that control the entire messaging process in AMQP message brokers, like RabbitMQ

To generate the secrets required for rabbitMQ, where n is the number of client certificates needed, run gen_cert.sh with the following arguments

sh cert-gen/gen_cert.sh -s=rabbit -a=rabbitmq-server.default.svc.cluster.local -n=2

This will generate the Kubernetes manifests rabbitmq-client-certs.yaml, rabbitmq-client-1-certs.yaml, rabbitmq-client-2-certs.yaml and rabbitmq-server-certs.yaml.

NOTE: Each instance of SPK in a cluster should be deployed with its own client certificate, so remember to generate enough initially.

SPK will still continue to work if all the RabbitMQ clients use the same client certificate but it is best if they use different ones, for

1. Better security

2. A customer site where different SPK instances may be using different certificates

3. Integration with a feature in a later release called Certificate Manager which rotates certificates and simplifies the process of obtaining, renewing and using such certificates

The first instance of SPK will then use rabbitmq-client-1-certs.yaml, the next rabbitmq-client-2-certs.yaml etc since rabbitmq-client-certs.yaml is reserved for CWC to use itself.

Deploy and License CWC in Disconnected Mode

CWC configured in “connected” mode performs License Activation and Verification without any manual intervention but CWC configured in “disconnected” mode requires the customer to intercept and manually proxy a bi-directional communication flow secured with a JWT.

The steps involved are given below.

1. Customer retrieves the JWT from MyF5 (Technical Services) for a given customer CAT and service.

2. Customer installs CWC using this JWT

3. Customer retrieves the initial configuration report from CWC using the CWC REST API. The first report is called the initial config report

4. Customer submits this report to TS

5. Customer retrieves a signed acknowledgement from TS for the configuration telemetry report and returns it to CWC.

6. CPCL verifies that this signed acknowledgement from TS relates to the report it previously issued

7. Customer installs SPK Controller

8. CPCL then notifies the SPK Controllers over RabbitMQ that they can start to process traffic.

9. The customer is then expected to periodically send updated telemetry reports to TS. Telemetry is generated monthly by CWC. If a customer fails to verify a previous month’s report, it will be rolled up into the next report.

Install CWC

JSON Web Token (JWT)

The RFC 7519 JSON Web Token (JWT) is a signed wrapper for the CAT which can be readily validated by the F5 API Portal. It is the first line of defense against denial-of-(database)-service attacks. The CAT provides secondary validation at the application level.

The customer will normally receive a JWT upon purchasing a subscription from MyF5.

A JWT is made up of 3 parts: Header, Payload, and Signature

Header

Claim	Description	Setting
kid	The "kid" (key ID) parameter is used to match a specific key in the JWK Set (JWKS)	First kid is "v1"
typ	The "typ" (type) Header Parameter is used to declare the media type.	Will always be "JWT" here in the JWT
alg	The "alg" (algorithm) parameter identifies the algorithm used to sign the token.	This will be set to "RS512" and the client should never accept a "none" value
jku	The "jku" (JWK Set URL) URI that refers to a resource for a set of JSON-encoded public keys, one of which corresponds to the key used to digitally sign the token	production tokens will point to the production URL

Payload

Claim	Description	Setting
sub	The "sub" (subject) claim identifies the principal that is the subject of the JWT	Contains the CAT
aud	The "aud" (audience) claim identifies the recipients that the JWT is intended for	Will be set to "urn:f5:teem
iat	The "iat" (issued at) claim identifies the time at which the JWT was issued	This is the date the subscription started
iss	The "iss" (issuer) claim identifies the principal that issued the JWT	Set to "F5 Inc."
jti	The "jti" (JWT ID) claim provides a unique identifier for the JWT	UUID
f5_order_type	The type of the JWT. Could be eval, freemium, paid, ela	Maps to the order type of the product entitlement
f5_order_subtype	Optional	Only set for a specific type of paid order

JSON Web Key (JWK)

The RFC 7517 JSON Web Key (JWK) is a JSON data structure that represents a set of public keys as a JSON object which can be used to verify a JWT

The ConfigMap cpcl-key-cm can be reconstructed from the following F5 JWK

curl -XGET https://product.apis.f5.com/ee/v1/keys/jwks --header 'Content-Type: application/json' | jq
{
  "keys": [
    {
      "kid": "v1",
      "alg": "RS512",
      "kty": "RSA",
      "n": "wgqDv-fuebdh_gV3wN8voRGcHGDo4YekYT78U2x-gAgxWDFFP4uIpQk9d_Hszevyr78xgFBD7RnR4FeWu7R62L1DnEEbrQYEjN..."

as shown here

oc get cm/cpcl-key-cm -n default -o yaml
apiVersion: v1
data:
  jwt.key: |
    {
      "keys": [
        {
          "kid": "v1",
          "alg": "RS512",
          "kty": "RSA",
          "n": "wgqDv-fuebdh_gV3wN8voRGcHGDo4YekYT78U2x-gAgxWDFFP4uIpQk9d_Hszevyr78xgFBD7RnR4FeWu7R62L1DnEEbrQYEjN..."                   

          ...

Helm Chart

Review the default values.yaml file which is packaged with the CWC Helm chart

cpclConfig:
  productName: "SPK"
  operationMode: "disconnected"
  signedVerificationCertPath: "/etc/cm20/cpcl/private/key/jwt.key"
  logLevel: "debug"
  jwt: ""
  digitalAssetName: "SPK"
  digitalAssetVersion: "1.2.0"
  friendlyName: "SPK Cluster"
  restApiPort: 30881
  
  ...

NOTE: The JWT must be specified before the Helm chart can be installed.

helm install spk f5ingress-dev/cwc --version 0.5.0 -n default --set cpclConfig.jwt="eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCIsImtpZCI6InYxIiwiamt1IjoiaHR0cHM6Ly9wcm9kdWN0LXRzdC5hcGlzLmY1bmV0d29ya3MubmV0L2VlL3YxL2tleXMvandrc..."
NAME: spk
LAST DEPLOYED: Mon Dec  5 01:36:36 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Cluster Wide Controller has been installed.

CWC reads the configuration to initialise CPCL from a ConfigMap which is created during this Helm installation.

oc get cm/cpcl-config-cm -n default -o yaml
apiVersion: v1
data:
  custom_id: SPK Cluster
  digital_asset_name: SPK
  digital_asset_version: 1.2.0

...

Access the CPCL API

To initially license and thereafter keep SPK licensed manually, the customer must be able to access the CPCL API and then manually upload the configuration report to F5.

By default CWC exposes a NodePort service on port 30881, so to access the API the customer must find an IP address for one of the SPK cluster nodes (oc get nodes -o wide) which can be reached from a jump box.

oc get svc/f5-spk-cwc -n default
NAME         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
f5-spk-cwc   NodePort   172.30.22.195   <none>        38081:30881/TCP   34m

NOTE: Since a customer may have security concerns about NodePort, there is an option in the values.yaml for the CWC HELM Chart to specify a ClusterIP instead. The customer must then use Openshift Routes to access the CWC service from outside the cluster.

Postman Collection

Curl is a more programmatic solution but this Postman Collection can also be used to license CWC.

Remember to add the CA certificate (cacert) under Settings

together with the relevant CPCL client certificate and key

License

To view details of the current active license

oc get cm/license-report -n peter-net -o yaml
apiVersion: v1
data:
  license-report: |
    appnamespaces:
    - pmills-app
    controllernamespace: peter-net
    numofspkpods: 1
    numofvcpus: 2
    usecases: []
kind: ConfigMap
metadata:
  creationTimestamp: "2022-12-09T13:17:23Z"
  name: license-report
  namespace: peter-net
  resourceVersion: "6342262"
  uid: 08c84d03-c81f-4af5-ba80-eb100846a53f

oc get secret licensestatus -n default -o jsonpath='{.data.licensestatus}{"\n"}' | base64 --decode | jq
{
"Name": "",
"State": 12,
"ErrCode": 0,
"ErrString": "",
"OperatorAction": "",
"DigitalAssetId": "1b377619-...",
"LicenseExpiryDate": "2023-12-04T00:01:00Z",
"EntitlementType": "paid",
"IsActive": true
}

State

Where the possible states are defined below. The /status API (below) shows them in a human readable string format

Description	State
StateCPCLDefault	0
StateCPCLInitialised	1
StateCPCLInitializationFailed	2
StateDeviceRegistrationInProgress	3
StateDeviceRegistrationComplete	4
StateDeviceRegistrationFailed	5
StateConfigReportInProgress	6
StateConfigReadyToDownload	7
StateConfigReportFailed	8
StateConfigReportDownloadFailed	9
StateConfigDownloaded	10
StateVerificationInProgress	11
StateVerificationComplete	12
StateVerificationFailed	13
StateExpiringSoon	14
StateExpiringToday	15
StateExpired	16
StateTelemetryInProgress	17
StateTelemetryReadyToGenerate	18

/status

curl -k --header 'Content-Type: application/json' \
--cert-type PEM \
--cacert /home/npi/api-server-secrets/ssl/ca/certs/ca_certificate.pem \
--cert /home/npi/api-server-secrets/ssl/client/certs/client_certificate.pem \
--key /home/npi/api-server-secrets/ssl/client/secrets/client_key.pem \
https://10.155.217.10:30881/status
{
  "Status": {
    "ClusterDetails": {
      "Name": "SPK Cluster"
    },
    "LicenseDetails": {
      "DigitalAssetID": "1b377619...",
      "EntitlementType": "paid",
      "LicenseExpiryDate": "2023-12-04T00:01:00Z",
      "LicenseExpiryInDays": "334"
    },
    "LicenseStatus": {
      "State": "Verification Complete"
    }
  },
  "TelemetryStatus": {
    "CurrentReport": {
      "StartDate": "2022-12-05 14:22:33.325423427 +0000 UTC m=+17151.693042043",
      "EndDate": "2022-12-31 14:22:36.409736702 +0000 UTC m=+2263554.777356511",
      "State": "Config Report Downloaded"
    },
    "NextReport": {
      "StartDate": "2023-01-01 00:00:00 +0000 UTC",
      "EndDate": "2023-01-31 00:00:00 +0000 UTC",
      "State": "Telemetry In Progress"
    }
  }
}

/report

Upon Initial Registration CWC will generate a Initial Config Report. Upon Switch License CWC generates an Updated Config Report and periodically CWC will generate a Telemetry Report. SPK is a postpaid service operating on a monthly billing cycle. The date a new telemetry report can be downloaded is determined by the EndDate of the NextReport in the status message. Prior to that day the following error message will be seen instead.

curl -k --header 'Content-Type: application/json' \
--cert-type PEM \
--cacert /home/npi/api-server-secrets/ssl/ca/certs/ca_certificate.pem \
--cert /home/npi/api-server-secrets/ssl/client/certs/client_certificate.pem \
--key /home/npi/api-server-secrets/ssl/client/secrets/client_key.pem \
https://10.155.217.10:30881/report
Cannot download config report in current state. Error Invalid State

Below is an actual report

curl -k --header 'Content-Type: application/json' \
> --cert-type PEM \
> --cacert /home/npi/api-server-secrets/ssl/ca/certs/ca_certificate.pem \
> --cert /home/npi/api-server-secrets/ssl/client/certs/client_certificate.pem \
> --key /home/npi/api-server-secrets/ssl/client/secrets/client_key.pem \
> https://10.155.217.10:30881/report
{"report":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJwYXlsb2FkIjp7ImRvY3VtZW50VHlwZSI6IlNQSyBUZWxlbWV0cnkiLCJkb2N1bWVudFZlcnNpb24iOiIxIiwiZGlnaXRhbEFzc..."}

The telemetry can be seen by copying and pasting the above report between the quotes into the following JWT decoding tool

Note the presence of the digitalAssetID field in the output discussed earlier.

/ee/v1/entitlements/telemetry

This report must then be uploaded to F5. The F5-DigitalAssetId is copied from the status report and the Authorization Bearer is copied from the JWT originally used to provision the CWC helm chart.

F5-TraceId is a globally unique identifier generated by the calling application which can be used to uniquely trace a request across all back-end systems. A new identifier should be generated and used for each request.

curl --location --request POST 'https://product.apis.f5.com/ee/v1/entitlements/telemetry' \
> --header 'F5-DigitalAssetId: 1b377619-c357-409b-97c0-923b688819e2' \
> --header 'User-Agent: SPK' \
> --header 'F5-TraceId: 350df9a4-4703-11ec-bbe5-acde48001122' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Bearer 
eyJhbGc...' \
> --data-raw '{"report": "eyJhbGci

etc.

The telemetry report response (manifest key) is a JWS Compact Serialization formatted document, digitally signed with a private key.

{"manifest":"eyJhbGciOiJSUzUxMiIsImtpZCI6InYxIiwiamt1IjoiaHR0cHM6Ly9wcm9kdWN0LXRzdC5hcGlzLmY1bmV0d29ya3MubmV0L2VlL3YxL2tleXMvandrcyJ9...."}

Install SPK (F5Ingress) Controller

A Security Context Constraint (SCC) is an OpenShift resource that restricts a pod to a group of resources and is similar to the K8S security context resource. The primary purpose of both is to limit a pod’s access to the host environment.The namespace where f5ingress is installed must have privileged SCC

oc get scc | grep privileged
privileged                        true    ["*"]        RunAsAny    RunAsAny           RunAsAny    RunAsAny    <no value>   false            ["*"]
#
oc get sa (service account)
NAME       SECRETS   AGE
builder    2         55d
default    2         55d
deployer   2         55d
#
oc adm policy add-scc-to-user privileged -n peter-net -z default
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:privileged added: "default"

helm install f5ingress f5ingress-dev/f5ingress --version 7.0.13 -n peter-net --values values.yaml
NAME: f5ingress
LAST DEPLOYED: Fri Dec 9 05:16:34 2022
NAMESPACE: peter-net
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The F5Ingress Controller has been installed.
TMM debug sidecar is deployed. To access: kubectl exec -it deployment/f5-tmm -c debug -n peter-net -- bash

Some of the default values in the values.yaml packaged with this helm chart must be overridden for the chart to be installed correctly.

1. Copy and paste the base64 encoded rabbitmq secrets from rabbitmq-client-1-certs.yaml into the values.yaml for the first SPK license helper container which talks to CWC. The dash characters (-) must be replaced by underscore characters (_), and the .pem suffix removed from the SSL/TLS certificate names

2. Specify the correct Kubernetes Network Attachment Definitions to use for your environment (cniNetworks)

3. Specify an application namespace to watch

4. Fortunately cwcNamespace already defaults to “default” and cpclConfig.operationMode to “disconnected” mode

f5_lic_helper:
    enabled: true
    name: f5-lic-helper
    cwcNamespace: default
    rabbitmqCerts:
      ca_root_cert: LS0...
      client_cert: LS0...
      client_key: LS0...