dSSM Database

Overview

The Cloud-Native Network Functions (CNFs) distributed Session State Management (dSSM) Pods provide centralized and persistent storage for the Traffic Management Microkernel (TMM) Proxy Pods. The dSSM Pods are Redis data structure stores that maintain application traffic data such as DNS/NAT46 translation mappings. The dSSM Pods bind to Kubernetes persistence volumes to persist data in the event of a container restart.

This document describes the dSSM Pods, and guides you through configuring and installing the f5-dssm-sentinel and f5-dssm-db containers.

_images/spk_info.png Note: To upgrade the dSSM databases and preserve all persisted data, review the Upgrading dSSM guide.

Sentinels and DBs

The dSSM Pods integrate as a StatefulSet, containing three dSSM Sentinel Pods and three dSSM DB Pods to maintain high availability. The Sentinel Pods elect and monitor a primary dSSM DB Pod, and if the primary dSSM DB Pod fails, a secondary DB will assume the primary role.

Additional high availability

The dSSM Pods also use the standard Kubernetes node affinity and PodDisruptionBudget features to maintain additional levels of high availability.

Affinity

Each dSSM Sentinel and DB Pod schedules onto a unique cluster node by default. The dSSM scheduling behavior can be modified using the dSSM Helm affinity_type parameter:

Setting Description
required Ensures the target cluster node does not currently host a Pod with the app=f5-dssm-db annotation (default).
preferred Attempt to schedule Pods onto unique nodes, but two dSSM Pods may schedule onto a single node when no schedulable nodes exists.
custom Scheduling behavior may be tuned specifically to the cluster admins requirements using the dSSM values.yaml file.

Helm parameter examples:

sentinel:
  affinity_type: "required"

db:
  affinity_type: "required"

_images/spk_info.png Kubernetes Assigning Pods overview.

PodDisruptionBudget

A minimum of 2 dSSM Pods remain available at all times based on the dSSM Helm pod_disruption_budget parameter. This parameter blocks voluntary interruptions to the dSSM Pod’s Running status. For example, if three schedulable nodes are available, and the admin runs oc adm drain on two of nodes in quick succession, the second action will be blocked until another schedulable node is added to the cluster.

Helm parameter examples:

sentinel:
  pod_disruption_budget:
    min_available: 2

db:
  pod_disruption_budget:
    min_available: 2

_images/spk_info.png Kubernetes Disruptions overview.

Sentinel Service

After installing dSSM, a dSSM Sentinel Service is created that receives data from TMM on TCP service port 26379, and forwards to the dSSM DB Pods using the same service port number. Ensure the Service port is available, and the cluster has CoreDNS enabled. In this example, the CNFs components will need to resolve the f5-dssm-sentinel.cnf-gateway hostname.

Example dSSM Service:

Name:              f5-dssm-sentinel
Namespace:         cnf-gateway
IP:                10.106.99.127
Port:              sentinel  26379/TCP
Endpoints:         10.244.1.15:26379,10.244.1.20:26379,10.244.4.3:26379

Requirements

Ensure you have:

Installation

Use the following steps to install the dSSM DB and Sentinel Pods.

  1. Change into the directory containing the latest CNFs Software, and obtain the f5-dssm Helm chart version:

    In this example, the CNFs files are in the cnfinstall directory:

    cd cnfinstall
    
    ls -1 tar | grep f5-dssm
    

    The example output should appear similar to the following:

    f5-dssm-1.2.0.tgz
    
  2. If you haven’t already, create a new Project for the CNFs Pods using the following command syntax:

    oc create ns <project name>
    

    In this example, a new Project named cnf-gateway is created:

    oc create ns cnf-gateway
    
  3. Generate the mTLS Secret for the TMM and dSSM communication channels:

    Note: The mTLS certificate can take up to 1 minute to generate.

    openssl dhparam -out dhparam2048.pem 2048
    openssl base64 -A -in dhparam2048.pem -out dhparam2048-encode.pem
    echo "apiVersion: v1" > dssm-certs.yaml
    echo "kind: Secret" >> dssm-certs.yaml
    echo "metadata:" >> dssm-certs.yaml
    echo " name: dssm-certs-secret" >> dssm-certs.yaml
    echo "data:" >> dssm-certs.yaml
    echo " dhparam2048.pem: `cat dhparam2048-encode.pem`" >> dssm-certs.yaml
    
  4. Add the dSSM serviceAccount to the Project’s privileged security context constraint (SCC):

    _images/spk_info.png Note: The f5-dssm serviceAccount name is based on the Helm release name. See Step 6.

    oc adm policy add-scc-to-user privileged -n <project> -z <serviceaccount>
    

    In this example, the f5-dssm serviceAccount is added to the cnf-gateway Project’s privileged SCC:

    oc adm policy add-scc-to-user privileged -n cnf-gateway -z f5-dssm
    
  5. Install the Secret to the dSSM Project:

    In this example, the Secrets install to the cnf-gateway namespace:

    oc apply -f dssm-certs.yaml -n cnf-gateway
    

    The command response should state the Secrets have been created:

    secret/dssm-certs-secret created
    
  6. Create a Helm values file named dssm-values.yaml, and set the image.repository parameters:

    In this example, Helm pulls the f5-dssm-store images from registry.com:

    image:
      repository: registry.com
    
    sentinel:
      fluentbit_sidecar:
        image:
          repository: registry.com
    
    db:
      fluentbit_sidecar:
        image:
          repository: registry.com
    
    cert_client_sidecar:
      image:
        repository: registry.com
    
  7. Required: If you deployed the Fluentd Logging Pod, you can send logging data to the f5-fluentd container by adding the fluentd.host parameters to the values file:

    sentinel:
      fluentbit_sidecar:
        fluentd:
          host: '<fluentd hostname>'
    
    db:
      fluentbit_sidecar:
        fluentd:
          host: '<fluentd hostname>'
    

    In this example, the Fluentd container is deployed to the cnf-gateway namespace:

    sentinel:
      fluentbit_sidecar:
        fluentd:
          host: 'f5-toda-fluentd.cnf-gateway.svc.cluster.local.'
    
    db:
      fluentbit_sidecar:
        fluentd:
          host: 'f5-toda-fluentd.cnf-gateway.svc.cluster.local.'
    
  8. Install the dSSM Pods using the following command syntax:

    helm install <release> tar/<helm-chart>.tgz \
    -f <values>.yaml -n <namespace>
    

    For example:

    helm install f5-dssm tar/f5-dssm-1.2.0.tgz \
    -f dssm-values.yaml -n cnf-gateway
    
  9. All dSSM Pods will be available after the election process, which can take up to a minute.

    _images/spk_warn.png Important: DB entries may fail to be created during the election process if TMM installs prior to completion. TMM will connect after the process completes.

    oc get pods -n cnf-gateway
    

    In this example, the dSSM Pods in the cnf-gateway namespace have completed the election process, and the Pod STATUS is Running:

    NAME                              READY   STATUS   
    f5-dssm-db-0                      3/3     Running   
    f5-dssm-db-1                      3/3     Running   
    f5-dssm-db-2                      3/3     Running   
    f5-dssm-sentinel-0                3/3     Running   
    f5-dssm-sentinel-1                3/3     Running   
    f5-dssm-sentinel-2                3/3     Running  
    
  10. The dSSM DB Pods should be bound to the persistent volumes:

    oc get pvc -n cnf-gateway
    

    In this example, the dSSM Pod’s PVC STATUS is Bound:

    NAME                STATUS   VOLUME                                     
    data-f5-dssm-db-0   Bound    pvc-c7060354-64d2-456b-9328-aa38f19b44b5   
    data-f5-dssm-db-1   Bound    pvc-8358b993-bf21-4fd7-a0fa-ee84ec420aac  
    data-f5-dssm-db-2   Bound    pvc-de65ed0f-f616-4021-a158-e0e78ed4539e   
    
  11. To securely connect the TMM and dSSM Pods, the CNFs Controller Helm values file requires the following paramaters:

    _images/spk_warn.png Important: Set the SESSIONDB_EXTERNAL_SERVICE parameter to the namespace of the dSSM Pod. In this example, cnf-utilities.

    tmm:
      sessiondb:
        useExternalStorage: "true"
    
      customEnvVars:
       - name: SESSIONDB_DISCOVERY_SENTINEL
         value: "true"
       - name: SESSIONDB_EXTERNAL_SERVICE
         value: "f5-dssm-sentinel.cnf-gateway"
       - name: SSL_SERVERSIDE_STORE
         value: "/tls/tmm/mds/clt"
       - name: SSL_TRUSTED_CA_STORE
         value: "/tls/tmm/mds/clt"
    
  12. Once the CNFs Controller has installed, verify connectivity between the dSSM and TMM Pods:

    oc logs f5-tmm-bcfd54cb9-tfqqv -c f5-tmm -n cnf-gateway | grep 'server successful'
    

    In this example, the dSSM and TMM Pods have successfully connected.

    Feb 04 21:24:41.193114 tmm[36]: redis_sentinel_connected/1004: Connection establishment with REDIS SENTINEL server successful
    

Next step

Continue to the CNFs CWC installation guide.

Feedback

Provide feedback to improve this document by emailing spkdocs@f5.com.