Introduction to ratio in MultiCluster

CIS supports distribution of traffic across clusters as per the cluster ratio configured for each cluster in the extended ConfigMap. To enable this feature, user defines ratio for clusters and sets the mode to ratio in the extended ConfigMap. When CIS posts the configurations to BIG-IP, it configures the traffic distribution weights for the service pools based on the defined ratios.

Note

Cluster ratio is not same as the Pool Member ratio that BigIP directly supports to configure for load balancing. However, it is handled through iRules and DataGroups that CIS creates according to the user defined ratio values.

Supported resources for cluster ratio configuration

Resource Cluster Ratio support
NextGenRoutes Yes
Virtual Server CR Yes
Transport Server CR No

Cluster ratio is supported with both High Availability (HA) CIS and standalone CIS. In both the cases, the mode must be set to ratio in the extended configMap.

However, the only configuration difference is that, for HA CIS, we define the Primary and Secondary cluster configurations. This enables the Cluster ratio to be defined as part of cluster configurations. In case of standalone CIS, since there are no Primary/Secondary cluster configurations, CIS relies on the localClusterRatio parameter to determine the Cluster ratio of the local cluster (cluster where the CIS is running).

Cluster ratio configuration

The cluster ratio should be defined in the extended configMap.

Note

  • Mode must be set to ratio which enables CIS distribute traffic based on cluster ratio.
  • Setting the ratio value is optional. If the ratio is not set, then 1 is considered default value.

Below is an example of multi-cluster configs, with HA CIS and ratio, in an extended ConfigMap.

CIS args:

  • --multi-cluster-mode=primary

or

  • --multi-cluster-mode=secondary
extendedSpec: |
  mode: ratio       -------------------------------------------|---> ratio needs to be set  |
  highAvailabilityCIS:   --------------------------------------|----------------------------|
    primaryEndPoint: http://10.145.72.114:8001                 |                            |
    probeInterval: 30                                          |                            |
    retryInterval: 3                                           |                            |
    primaryCluster:  ------------------------------------------|----------------------------|
      clusterName: cluster1                                    |  Ratio = 1 (Default Value) |
      secret: default/kubeconfig1 -----------------------------|----------------------------|
    secondaryCluster:                                          |                            |
      clusterName: cluster2                                    |                            |
      secret: default/kubeconfig2                              |
      ratio: 3  -----------------------------------------------|---> Ratio = 3              |
  externalClustersConfig:    ----------------------------------|----------------------------|
  - clusterName: cluster3                                      |                            |
    secret: default/kubeconfig3
    ratio: 5    -----------------------------------------------|---> Ratio = 5              |
  - clusterName: cluster4    ----------------------------------|----------------------------|
    secret: default/kubeconfig4                                |  Ratio = 1 (Default Value) |

Below is an example of multi-cluster configs, with standalone CIS and ratio, in an extended ConfigMap.

CIS args:

  • --multi-cluster-mode=standalone
extendedSpec: |
  mode: ratio       -------------------------------------------|---> ratio needs to be set  |
  localClusterRatio: 2    -------------------------------------|---> Ratio = 2              |
  externalClustersConfig:    ----------------------------------|----------------------------|
  - clusterName: cluster3                                      |                            |
    secret: default/kubeconfig3
    ratio: 5    -----------------------------------------------|---> Ratio = 5              |
  - clusterName: cluster4    ----------------------------------|----------------------------|
    secret: default/kubeconfig4                                |  Ratio = 1 (Default Value) |

Calculating traffic distribution based on defined ratio

Note

This example illustrates the combination of cluster ratio along with a route that has A/B or Alternate Backends. However, it works the same way if you have cluster ratio and a simple route without any alternate backends (no weights specified for the services).

Assuming we have four K8S/Openshift clusters (CL1, CL2, CL3, CL4), with the following defined ratios for each cluster:

ResourcCluster Name Ratio
CL1 4
CL2 3
CL3 2
CL4 1

Let us consider that we have created the following NextGenRoute resource:

Route.yaml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
 apiVersion: v1
 kind: Route
 metadata:
   name: route-example
   namespace: foo
   annotations:
     virtual-server.f5.com/multiClusterServices:  |
       [
         {
           "clusterName": "cluster3",
           "service": "svc1",
           "namespace": "foo",
           "servicePort": 80,
           "weight": 20,
         },
         {
           "clusterName": "cluster3",
           "service": "svc2",
           "namespace": "foo",
           "servicePort": 80,
           "weight": 10,
         }
       ]
 spec:
   host: www.example.com
   to:
     kind: Service
     name: baseSvc
     weight: 30
   alternateBackends:
   - kind: Service
     name: AbSvc1
     weight: 10
   - kind: Service
     name: AbSvc2
     weight: 20

The calculation for traffic distribution is as follows:

  • Effective weight for a service(S) = Ws/Wt * Rc/Rt

    Ws => Weight specified for the service

    Wt => Sum of weights of all services (VS service + Alternate backends + External services)

    Rc => Ratio specified for the cluster on which the service is running

    Rt => Sum of all the ratios of the clusters excluding those cluster ratios which do not contribute to this Route services

  • Considering the above example, the services associated with the route are as follows:

    Route(P) (Route in primary cluster) => Associated services are (baseSvc(P), AbSvc1(P), AbSvc2(P), svc1 and svc2)

    Route(S) (Route in secondary cluster) => Associated services are (baseSvc(S), AbSvc1(S), AbSvc2(S), svc1 and svc2)

    ‘*’ Where (P) and (S) stand for primary and secondary cluster

  • There are 4 clusters CL1, CL2, CL3, CL4 and ratios defined for these clusters along with the service weights are:

    CL1(Primary) => Ratio: 4 ([baseSvc(P) => weight 30 ] + Alternate backend services [ AbSvc1(P) => weight:10, AbSvc2(P) => weight:20 ])

    CL2(Secondary) => Ratio: 3 ([baseSvc(S) => weight 30 ] + Alternate backend services [ AbSvc1(S) => weight:10, AbSvc2(S) => weight:20 ])

    CL3 => Ratio: 2 ([svc1 => weight 20], [svc2 => weight 10])

    CL4 => Ratio: 1 (No services)

  • Effective weight calculation considering the service weights as well as the cluster ratio:

    Total Weight(Wt) = 30[baseSvc(P)] + 30[baseSvc(S)] + 10[AbSvc1(P)] + 10[AbSvc1(S)] + 20[AbSvc2(P)] + 20[AbSvc2(S)] + 20(svc1) + 10(svc2) = 150

    Total Ratio(Rt) = 4(CL1) + 3(CL2) + 2(CL3) = 9 [Excluded CL4 ratio as it doesn’t contribute to the Route’s services]

    Effective weight for service baseSvc(P) : 30(Ws)/150(Wt) * 4(Rc)/9(Rt) = 3/15 * 4/9 = 0.088

    Effective weight for service baseSvc(S) : 30(Ws)/150(Wt) * 3(Rc)/9(Rt) = 3/15 * 3/9 = 0.066

    Effective weight for service AbSvc1(P) : 10(Ws)/150(Wt) * 4(Rc)/9(Rt) = 1/15 * 4/9 = 0.029

    Effective weight for service AbSvc1(S) : 10(Ws)/150(Wt) * 3(Rc)/9(Rt) = 1/15 * 3/9 = 0.022

    Effective weight for service AbSvc2(P) : 20(Ws)/150(Wt) * 4(Rc)/9(Rt) = 2/15 * 4/9 = 0.059

    Effective weight for service AbSvc2(S) : 20(Ws)/150(Wt) * 3(Rc)/9(Rt) = 2/15 * 3/9 = 0.044

    Effective weight for service svc1 : 20(Ws)/150(Wt) * 2(Rc)/9(Rt) = 2/15 * 2/9 = 0.029

    Effective weight for service svc2 : 10(Ws)/150(Wt) * 2(Rc)/9(Rt) = 1/15 * 2/9 = 0.014

  • Finally, after considering the services associated with the particular Route, cluster ratios and service weights, we have calculated the effective weights for each service. CIS will use these weights to decide traffic distribution, as mentioned below:

    Total Weight = 0.088 + 0.066 + 0.029 + 0.022 + 0.059 + 0.044 + 0.029 + 0.014 = 0.351

    Traffic distribution for a service = (Effective weight for the service / Total effective weight) * 100 %

    Final traffic distribution:

    service baseSvc(P) : (0.088/0.351) % 100 = 25.07% ~ 25%

    service baseSvc(S) : (0.066/0.351) % 100 = 18.8% ~ 19%

    service ABs1(P) : (0.029/0.351) % 100 = 8.2% ~ 8%

    service ABs1(S) : (0.022/0.351) % 100 = 6.2% ~ 6%

    service ABs2(P) : (0.059/0.351) % 100 = 16.8% ~ 17%

    service ABs2(S) : (0.044/0.351) % 100 = 12.5% ~ 13%

    service svc1 : (0.029/0.351) % 100 = 8.2% ~ 8%

    service svc2 : (0.014/0.351) % 100 = 3.9% ~ 4%

The following diagram shows the effective traffic distribution:

../../_images/cluster-ratio-wise-traffic-distribution.png

Does a cluster without any hosted service contribute to the ratio-based traffic distribution?

The clusters that do not host any service do not affect the ratio-based traffic distribution. This means that the ratio of clusters that have not hosted any services connected to the NextGenRoute/VS CR resource will not be considered while calculating the final ratio for traffic distribution.

Does it work along with Alternate Backends?

Ratio based traffic distribution works even along with Alternate Backends where different weights are defined for each service. In such a scenario the ratio of traffic distribution is computed taking into consideration both the service weights and cluster ratio. For more information, see A/B or Alternate Backends.