Introduction to ratio in MultiCluster¶
CIS supports distribution of traffic across clusters as per the cluster ratio configured for each cluster in the extended ConfigMap. To enable this feature, user defines ratio for clusters and sets the mode to ratio in the extended ConfigMap. When CIS posts the configurations to BIG-IP, it configures the traffic distribution weights for the service pools based on the defined ratios.
Note
Cluster ratio is not same as the Pool Member ratio that BigIP directly supports to configure for load balancing. However, it is handled through iRules and DataGroups that CIS creates according to the user defined ratio values.
Supported resources for cluster ratio configuration¶
| Resource | Cluster Ratio support |
|---|---|
| NextGenRoutes | Yes |
| Virtual Server CR | Yes |
| Transport Server CR | No |
Cluster ratio is supported with both High Availability (HA) CIS and standalone CIS. In both the cases, the mode must be set to ratio in the extended configMap.
However, the only configuration difference is that, for HA CIS, we define the Primary and Secondary cluster configurations. This enables the Cluster ratio to be defined as part of cluster configurations. In case of standalone CIS, since there are no Primary/Secondary cluster configurations, CIS relies on the localClusterRatio parameter to determine the Cluster ratio of the local cluster (cluster where the CIS is running).
Cluster ratio configuration¶
The cluster ratio should be defined in the extended configMap.
Note
- Mode must be set to ratio which enables CIS distribute traffic based on cluster ratio.
- Setting the ratio value is optional. If the ratio is not set, then 1 is considered default value.
Below is an example of multi-cluster configs, with HA CIS and ratio, in an extended ConfigMap.
CIS args:
--multi-cluster-mode=primary
or
--multi-cluster-mode=secondary
extendedSpec: |
mode: ratio -------------------------------------------|---> ratio needs to be set |
highAvailabilityCIS: --------------------------------------|----------------------------|
primaryEndPoint: http://10.145.72.114:8001 | |
probeInterval: 30 | |
retryInterval: 3 | |
primaryCluster: ------------------------------------------|----------------------------|
clusterName: cluster1 | Ratio = 1 (Default Value) |
secret: default/kubeconfig1 -----------------------------|----------------------------|
secondaryCluster: | |
clusterName: cluster2 | |
secret: default/kubeconfig2 |
ratio: 3 -----------------------------------------------|---> Ratio = 3 |
externalClustersConfig: ----------------------------------|----------------------------|
- clusterName: cluster3 | |
secret: default/kubeconfig3
ratio: 5 -----------------------------------------------|---> Ratio = 5 |
- clusterName: cluster4 ----------------------------------|----------------------------|
secret: default/kubeconfig4 | Ratio = 1 (Default Value) |
Below is an example of multi-cluster configs, with standalone CIS and ratio, in an extended ConfigMap.
CIS args:
--multi-cluster-mode=standalone
extendedSpec: |
mode: ratio -------------------------------------------|---> ratio needs to be set |
localClusterRatio: 2 -------------------------------------|---> Ratio = 2 |
externalClustersConfig: ----------------------------------|----------------------------|
- clusterName: cluster3 | |
secret: default/kubeconfig3
ratio: 5 -----------------------------------------------|---> Ratio = 5 |
- clusterName: cluster4 ----------------------------------|----------------------------|
secret: default/kubeconfig4 | Ratio = 1 (Default Value) |
Calculating traffic distribution based on defined ratio¶
Note
This example illustrates the combination of cluster ratio along with a route that has alternate backends. However, it works the same way if you have cluster ratio and a simple route without any alternate backends (no weights specified for the services).
Assuming we have four K8S/Openshift clusters (CL1, CL2, CL3, CL4), with the following defined ratios for each cluster:
| ResourcCluster Name | Ratio |
|---|---|
| CL1 | 4 |
| CL2 | 3 |
| CL3 | 2 |
| CL4 | 1 |
Let us consider that we have created the following NextGenRoute resource:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | apiVersion: v1 kind: Route metadata: name: route-example namespace: foo annotations: virtual-server.f5.com/multiClusterServices: | [ { "clusterName": "cluster3", "service": "svc1", "namespace": "foo", "servicePort": 80, "weight": 20, }, { "clusterName": "cluster3", "service": "svc2", "namespace": "foo", "servicePort": 80, "weight": 10, } ] spec: host: www.example.com to: kind: Service name: baseSvc weight: 30 alternateBackends: - kind: Service name: AbSvc1 weight: 10 - kind: Service name: AbSvc2 weight: 20 |
The calculation for traffic distribution is as follows:
Effective weight for a service(S) = Ws/Wt * Rc/Rt
Ws => Weight specified for the service
Wt => Sum of weights of all services (VS service + Alternate backends + External services)
Rc => Ratio specified for the cluster on which the service is running
Rt => Sum of all the ratios of the clusters excluding those cluster ratios which do not contribute to this Route services
Considering the above example, the services associated with the route are as follows:
Route(P) (Route in primary cluster) => Associated services are (baseSvc(P), AbSvc1(P), AbSvc2(P), svc1 and svc2)
Route(S) (Route in secondary cluster) => Associated services are (baseSvc(S), AbSvc1(S), AbSvc2(S), svc1 and svc2)
‘*’ Where (P) and (S) stand for primary and secondary cluster
There are 4 clusters CL1, CL2, CL3, CL4 and ratios defined for these clusters along with the service weights are:
CL1(Primary) => Ratio: 4 ([baseSvc(P) => weight 30 ] + Alternate backend services [ AbSvc1(P) => weight:10, AbSvc2(P) => weight:20 ])
CL2(Secondary) => Ratio: 3 ([baseSvc(S) => weight 30 ] + Alternate backend services [ AbSvc1(S) => weight:10, AbSvc2(S) => weight:20 ])
CL3 => Ratio: 2 ([svc1 => weight 20], [svc2 => weight 10])
CL4 => Ratio: 1 (No services)
Effective weight calculation considering the service weights as well as the cluster ratio:
Total Weight(Wt) = 30[baseSvc(P)] + 30[baseSvc(S)] + 10[AbSvc1(P)] + 10[AbSvc1(S)] + 20[AbSvc2(P)] + 20[AbSvc2(S)] + 20(svc1) + 10(svc2) = 150
Total Ratio(Rt) = 4(CL1) + 3(CL2) + 2(CL3) = 9 [Excluded CL4 ratio as it doesn’t contribute to the Route’s services]
Effective weight for service baseSvc(P) : 30(Ws)/150(Wt) * 4(Rc)/9(Rt) = 3/15 * 4/9 = 0.088
Effective weight for service baseSvc(S) : 30(Ws)/150(Wt) * 3(Rc)/9(Rt) = 3/15 * 3/9 = 0.066
Effective weight for service AbSvc1(P) : 10(Ws)/150(Wt) * 4(Rc)/9(Rt) = 1/15 * 4/9 = 0.029
Effective weight for service AbSvc1(S) : 10(Ws)/150(Wt) * 3(Rc)/9(Rt) = 1/15 * 3/9 = 0.022
Effective weight for service AbSvc2(P) : 20(Ws)/150(Wt) * 4(Rc)/9(Rt) = 2/15 * 4/9 = 0.059
Effective weight for service AbSvc2(S) : 20(Ws)/150(Wt) * 3(Rc)/9(Rt) = 2/15 * 3/9 = 0.044
Effective weight for service svc1 : 20(Ws)/150(Wt) * 2(Rc)/9(Rt) = 2/15 * 2/9 = 0.029
Effective weight for service svc2 : 10(Ws)/150(Wt) * 2(Rc)/9(Rt) = 1/15 * 2/9 = 0.014
Finally, after considering the services associated with the particular Route, cluster ratios and service weights, we have calculated the effective weights for each service. CIS will use these weights to decide traffic distribution, as mentioned below:
Total Weight = 0.088 + 0.066 + 0.029 + 0.022 + 0.059 + 0.044 + 0.029 + 0.014 = 0.351
Traffic distribution for a service = (Effective weight for the service / Total effective weight) * 100 %
Final traffic distribution:
service baseSvc(P) : (0.088/0.351) % 100 = 25.07% ~ 25%
service baseSvc(S) : (0.066/0.351) % 100 = 18.8% ~ 19%
service ABs1(P) : (0.029/0.351) % 100 = 8.2% ~ 8%
service ABs1(S) : (0.022/0.351) % 100 = 6.2% ~ 6%
service ABs2(P) : (0.059/0.351) % 100 = 16.8% ~ 17%
service ABs2(S) : (0.044/0.351) % 100 = 12.5% ~ 13%
service svc1 : (0.029/0.351) % 100 = 8.2% ~ 8%
service svc2 : (0.014/0.351) % 100 = 3.9% ~ 4%
The following diagram shows the effective traffic distribution:
Does a cluster without any hosted service contribute to the ratio-based traffic distribution?
The clusters that do not host any service do not affect the ratio-based traffic distribution. This means that the ratio of clusters that have not hosted any services connected to the NextGenRoute/VS CR resource will not be considered while calculating the final ratio for traffic distribution.
Does it work along with Alternate Backends?
Ratio based traffic distribution works even along with Alternate Backends where different weights are defined for each service. In such a scenario the ratio of traffic distribution is computed taking into consideration both the service weights and cluster ratio.