SPK CWC

Overview

The Service Proxy for Kubernetes (SPK) Cluster Wide Controller (CWC) enables SPK’s software licensing and billing capabilities. Once the SPK software is installed and licensed, the CWC collects and reports software usage telemetry statistics for each of the SPK Controller instances in the cluster. SPK uses F5’s flexible consumption software licensing model, billing only for the SPK features used.

_images/spk_info.png Note: SPK Licensing applies to the cluster level, and is performed prior to installing the SPK Controller instances.

This document guides you through installing the CWC controller.

CPCL module

The CWC contains the Common Product Component and Libraries (CPCL) module that helps with license activation, and with generating and maintaining the monthly license reports. The CPCL requires an SSL/TLS certificate, and an F5 provided SSL/TLS key and unique JSON Web Token (JWT) to identify the cluster. Installing the CPCL SSL/TLS certificate and key will be demonstrated later in this overview, and the license reporting will demonstrated in the SPK Licensing overview.

Cluster Project

The CWC Pod can install to any cluster Project. In this document, the CWC will install to the spk-telemetry Project.

RabbitMQ

The CWC uses the RabbitMQ open source message broker to integrate with the SPK Controller Pod(s). Ensure connectivity is allowed for the service ports listed below.

CWC Service

After installing the CWC, a CWC Service object is created that receives REST API data on TCP service port 30881, and forwards the data to the CWC Pod on TCP service port 38081. Ensure the Service ports are available, and the cluster has CoreDNS enabled. In this example, the SPK components will need to resolve the f5-spk-cwc.spk-telemetry DNS hostname.

Name:              f5-spk-cwc
Namespace:         spk-telemetry
IP:                10.109.102.215
Port:              cwc-rest  30881/TCP
Endpoints:         10.244.1.75:38081

RabbitMQ Service

After installing the CWC, a RabbitMQ Service object is created, to pass messages between the SPK Controllers and the CWC on TCP service port 5671. Ensure the Service port is available, and the cluster’s core DNS is enabled. In this example, the SPK components will need to resolve the rabbitmq-server.spk-telemetry hostname.

Name:              rabbitmq-server
Namespace:         spk-telemetry
IP:                10.109.105.210
Port:              ampqst  5671/TCP
Endpoints:         10.244.1.80:5671

CWC Capabilities

The CWC supports the following capabilities:

  • Licensing. Refer to the SPK Licensing guide for more information.
  • Debugging. Refer to the Debug API guide for more information.

Requirements

Ensure you have:

  • Installed the SPK software.
  • A Linux workstation with Helm, OpenSSL and make installed.
  • Obtained the CPCL SSL/TLS key and the JWT from your MyF5 account.

Procedures

Create cluster Secrets and CWC certificate

Use this procedure to create and install Kubernetes Secrets used to secure communication between the CWC, RabbitMQ and SPK Controller Pods, and create the SSL/TLS certificates required to authenticate the CWC REST API for licensing purposes.

_images/spk_info.png Note: F5 recommends obtainin certificate authority (CA) signed certificates using the Subject Alternative Names (SANs) shown with -a in steps 3 and 5.

  1. Change into local directory with the SPK Software files, and list the files in the tar directory:

    In this example, the SPK files are in the spkinstall directory.

    cd spkinstall
    
    ls -1 tar
    

    This procedure requires the f5-cert-gen-0.2.4.tgz file.

    csrc-0.1.4.tgz
    cwc-0.5.0.tgz
    f5-cert-gen-0.2.4.tgz
    f5-dssm-0.22.18.tgz
    f5-toda-fluentd-1.10.1.tgz
    f5ingress-7.0.13.tgz
    spk-docker-images.tgz
    
  2. Extract the cert-gen utility to generate Secrets and SSL/TLS certificates:

    tar xvf tar/f5-cert-gen-0.2.4.tgz
    
  3. Generate the Secret and the SSL/TLS certificates for the CWC REST API:

    _images/spk_info.png Note: The SSL/TLS certificates will be referenced in the Configure Postman section of the SPK Licensing guide.

    sh cert-gen/gen_cert.sh -s=api-server -a=f5-spk-cwc.<project> -n=1
    

    In this example, the CWC installs to the spk-telemetry Project.

    sh cert-gen/gen_cert.sh -s=api-server -a=f5-spk-cwc.spk-telemetry -n=1
    

    The command output indicates the Secret has been created:

    Generating /path/cwc-license-certs.yaml
    
  4. Install the CWC Secret:

    In this example, the CWC installs to the spk-telemetry Project.

    oc apply -f cwc-license-certs.yaml -n spk-telemetry
    

    The command output indicates the Secret was created successfully:

    secret/cwc-license-certs created
    
  5. Generate the client and server Secrets used to secure the RabbitMQ and CWC channel:

    _images/spk_info.png Note: Set the -n= option to the number of SPK Controller Pods to license, and add 1 for the CWC Pod. It’s okay to set a number allowing for future SPK Controller instances. The example below allows one CWC and two SPK controllers.

    sh cert-gen/gen_cert.sh -s=rabbit \
    -a=rabbitmq-server.<project>.svc.cluster.local \
    -n=3
    

    In this example, the CWC installs to the spk-telemetry Project.

    sh cert-gen/gen_cert.sh -s=rabbit \
    -a=rabbitmq-server.spk-telemetry.svc.cluster.local \
    -n=3
    

    The command output indicates the Secrets have been created.

    client1_certificate.pem
    client1_key.pem
    client2_certificate.pem
    client2_key.pem
    Generating /path/rabbitmq-server-certs.yaml
    Generating /path/rabbitmq-client-certs.yaml
    client1_certificate.pem
    client1_key.pem
    Generating /path/rabbitmq-client-1-certs.yaml
    client2_certificate.pem
    client2_key.pem
    Generating /path/rabbitmq-client-2-certs.yaml
    
  6. Install the client and server Secrets for the CWC and RabbitMQ channel:

    In this example, the CWC RabbitMQ client Secret installs to the spk-telemetry Project.

    oc apply -f rabbitmq-client-certs.yaml -n spk-telemetry
    
    secret/client-certs created
    

    In this example, the RabbitQM server Secret installs to the spk-telemetry Project.

    oc apply -f rabbitmq-server-certs.yaml -n spk-telemetry
    
    secret/server-certs created
    
  7. Continue to the next procedure.

Install the CPCL Certicate and Key

Use these steps to install SSL/TLS certificate and key used CWC to authentiate the CPCL module.

  1. To install the CPCL SSL/TLS certificate, copy the cpcl-crt-cm ConfigMap into a YAML file:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: cpcl-crt-cm
    data:
      jwt_ca.crt: |+
        -----BEGIN CERTIFICATE-----
        MIIDbzCCAlegAwIBAgIBATANBgkqhkiG9w0BAQsFADA1MQswCQYDVQQGEwJTRTEU
        MBIGA1UEChMLQ29tcGFueSBDby4xEDAOBgNVBAMTB1Jvb3QgQ0EwHhcNMjEwNzA1
        MTQzMzEzWhcNMzEwNzA1MTQzMzIzWjAxMQswCQYDVQQGEwJTRTEUMBIGA1UEChML
        Q29tcGFueSBDby4xDDAKBgNVBAMTA0RDQTCCASIwDQYJKoZIhvcNAQEBBQADggEP
        ADCCAQoCggEBAMlzVdnBKDTmZy6yCQ9qw9OyYWh0lq5nD126LFX2UyZbIR2sNrpt
        WiTLizaxA0snf24Ha3nSA8MWraxuh8p1x0IEF8J+FsOpCzSWlU3P1C1bThWnkmco
        aJx/dGMtNHMhHWJn8bowUKFmSFLGL3wYWZbjoRWHuwaW3P0WqGqTo82ttjQPhK7u
        RW/U0OP+G9tkZAJXGQdaJseO8Km8Sfvw62xUgG28GXOiL2nNLEW5Jqg5FB8Ib/dB
        RtclIte87nf9uK/5KOJadzdthQeFmrBUzizE5mQTtegUiHUaNrXDAWdeljD4HMCy
        Z47SoghEaDVuJwcaDKUxIfC1PtOQnCbmZ1kCAwEAAaOBjTCBijAOBgNVHQ8BAf8E
        BAMCAQYwEwYDVR0lBAwwCgYIKwYBBQUHAwEwEgYDVR0TAQH/BAgwBgEB/wIBATAd
        BgNVHQ4EFgQUFh1AknXyhoLd03dQppbVU3GAryowHwYDVR0jBBgwFoAUFzn9dWIf
        8WQzkjGqZs2jDKtk6TYwDwYDVR0RBAgwBocEfwAAATANBgkqhkiG9w0BAQsFAAOC
        AQEAkxBkFBuxvFCZL4/bWSlpHJKo7UCbcASzuMbdMThgf6OPYx+ggmuQZh3+DZ/4
        rTvf6YRrSYuceuF2c26tlknhT9uehYdz4Q/75RFzhwT4PvmUZ6agRJB5I9FsdjBN
        Q101ew1t6aPmoGPViiosEYVWIRf/0du/WycorNMh3WMo7cZ9+UuBkgehVYz0rxyO
        sOf0apgk+oLC04RmoUkVU5AVX/5xWSA0o++SHlv3tkKoCRooE/G7ke7ie18bjCr0
        laFS3U1i0dcEPMTvy0+kkwrkO/1onZRhzOTk1E7AsAlHlwe78p3g26JaZ3d+IzJM
        ommDCLNJvSoo3MUxEqVKsIgEvw==
        -----END CERTIFICATE-----
    
  2. Install the Certificate ConfigMap:

    In this example, the ConfigMap installs to the spk-telemetry Project:

    oc apply -f cpcl-cert.yaml -n spk-telemetry
    
  3. To install the CPCL SSL/TLS key, copy the cpcl-key-cm ConfigMap into a YAML file, and add the key data provided by F5 with the JWT:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: cpcl-key-cm
    data:
      jwt.key: |+
        <CPCL key>
    

    The example output has been shortened for readability.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: cpcl-key-cm
    data:
      jwt.key: |+
        {
          "keys": [
            {
              "kid": "v1",
              "alg": "RS512",
              "kty": "RSA",
              "n": "24FcB1269RC6WNgPghIB7X772zTTts0",
              "e": "AQAB",
              "x5c": [
                "MIIFdBCAABJClAwIRAK+LbrS2gmaJSeoUZ",
                "MIIFCjACAvbbagAwBAgBBIBTNBgkqhkiG8",
                "MIIJHADLLBOigAzIBAaIJAIozdNNO8kBMA",
                "MIIGFazBBD/+gAwIBAgITABANBgkqkhqq9",
              ],
              "use": "sig"
            }
          ]
        }
    
  4. Install the Key ConfigMap:

    In this example, the ConfigMap installs to the spk-telemetry Project:

    oc apply -f cpcl-key.yaml -n spk-telemetry
    
  5. Continue to the next procedure.

Install the CWC

Use these steps to install the CWC Pod to the spk-telemetry Project.

  1. Change into the directory with the SPK software files, and list the files in the tar directory:

    In this example, the SPK files are in the spkinstall directory:

    cd spkinstall
    
    ls -1 tar
    

    This procedure requires the cwc-0.5.0.tgz Helm chart.

    cwc-0.5.0.tgz
    f5-cert-gen-0.2.4.tgz
    f5-dssm-0.22.5.tgz
    f5-toda-fluentd-1.8.23.tgz
    f5ingress-5.0.16.tgz
    spk-docker-images.tgz
    
  2. Create a Helm values file named cwc-values.yaml, set the image.repository parameter value to the local image repository’s hostname or IP address:

    In this example, Helm pulls the CWC Pod images from local.registry.com.

    image:
      repository: "local.registry.com"
    
  3. Install the CWC Pod, and reference the JWT:

    helm install spk-cwc tar/cwc-0.5.0.tgz -f cwc-values.yaml \
    --set cpclConfig.jwt=<jwt> -n <project>
    

    In this example, the JWT has been truncated for readability, and installs to the spk-telemetry Project.

    helm install spk-cwc tar/cwc-0.5.0.tgz -f cwc-values.yaml \
    --set cpclConfig.jwt=eyJhbGciOiJSUzUxMiIsInR5cCI6 -n spk-telemetry
    
  4. The CWC Pod’s spk-cwc and rabbitmq-server containers should be in the Running state:

    oc get pods -n spk-telemetry | grep -E 'STATUS|f5-spk-cwc'
    
    NAME                          READY   STATUS    RESTARTS 
    f5-spk-cwc-68b5cf9565-zs6rg   2/2     Running   0       
    
  5. Continue to the next procedure.

Update the Controller values

Each SPK Controller installs to a unique Project, and will require its own set of RabbitMQ Secrets, generated previously with Install the Secrets. Use the following steps to add the RabbitMQ Secrets to each of the SPK Controller’s Helm values file.

_images/spk_info.png Note: The cluster will be licensed in the SPK Licensing procedure, followd by the SPK Controller installation procedure that will include these values.

  1. Cat the first (of two) RabbitMQ Secret files named rabbitmq-client-1-certs.yaml:

    cat rabbitmq-client-1-certs.yaml
    

    The example output has been shortened for readability.

    kind: Secret
    apiVersion: v1
    metadata:
     name: client-certs
    data:
     ca-root-cert.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk
     client-cert.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1
     client-key.pem: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1J
    
  2. Copy the three .pem SSL/TLS certificates listed beneath the data: parameter.

  3. Edit the SPK Controller’s Helm values file, and add the SSL/TLS certificates to the controller section. Ensure you modify the image.repository parameter for the local image registry, and the cwcNamespace for the Project the CWC installs to:

    _images/spk_warn.png Important: The dash characters (-) convert to underscore characters (_), and the .pem suffix is removed from the SSL/TLS certificate names.

    controller:
      f5_lic_helper:
        enabled: true
        cwcNamespace: <project>
        image:
          repository: "<local.registry.com>"
        rabbitmqCerts:
          ca_root_cert: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk
          client_cert: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1
          client_key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1J
    
  4. Repeat steps 1 - 3 using the subsequent SSL/TLS files. For example, use rabbitmq-client-2-certs.yaml to prepare the values for a second SPK Controller instance.

  5. Continue to the Next step section.

Next step

Continue to the SPK Licensing guide to license the cluster.

Feedback

Provide feedback to improve this document by emailing spkdocs@f5.com.

Supplemental