StaticRouteSupport¶
Support for CIS to configure static routes in the BIG-IP with node subnets assigned to the nodes in the OpenShift/k8s cluster. This enables direct routing from BIG-IP to k8s Pods in cluster mode without vxaln tunnel configuration on BIG-IP.
Configuration¶
To enable the static route configuration, set --static-routing-mode to true and --orchestration-cni to CNI configured in the cluster.
args:
--static-routing-mode=true
--orchestration-cni=<ovn-k8s/flannel/antrea/cilium-k8s>
With CNI ovn-k8s, if the node has multiple interfaces, --static-route-node-cidr can be configured to specify the node network from which nodeip has to be selected.
Without this config, CIS always picks the primary interface address from annotation k8s.ovn.org/node-primary-ifaddr on node manifest as nodeip for static route creation on the BIG-IP.
Use cis-deployment-ovn-k8s-mnic.yaml to deploy with this configuration.
args:
--static-route-node-cidr=10.4.0.0/14
Parameters for StaticRoutingMode¶
| Parameter | Type | Required | Default | Description | Allowed Values | Agent | Minimum Supported CIS Version |
|---|---|---|---|---|---|---|---|
| –static-routing-mode | Boolean | Optional | false | Adds Static Routes on the BIG-IP so traffic can be directly routed to the pods. (Without tunnels). | true, false | AS3 | 2.13.0 |
| orchestration-cni | String | Optional | flannel | Kubernetes cluster CNI name. | cilium-k8s, flannel, ovn-k8s, antrea | AS3 | 2.13.0 |
| shared-static-routes | Boolean | Optional | false | When set to true, static routes are created on the /Common partition, which can be valid only when static-routing-mode is enabled. | true, false | AS3 | 2.14.0 |
| static-route-node- cidr | String | Optional | N/A | Specify the node network cidr to be used for static routing when the node has multiple interfaces. This is supported only with CNI ovn-k8s | Any valid cidr eg: 10.4.0.0/14 | AS3 | 2.15.0 |
cis-deployment-ovn-k8s.yaml¶
By deploying cis-deployment-ovn-k8s.yaml in your cluster, CIS will be configured with static route enabled for the ovn-k8s environment.
The route will be configured with the name in the format k8s-<nodename>-<nodeip> as shown below.
BIG-IP Config¶
Validate the static routes added to the BIG-IP.
Note
- StaticRoutingMode is required only with cluster mode where vxlan tunnel is not configured.
- CIS uses –orchestration-cni to read node subnet info and nodeip based on the CNI configured.
Troubleshooting¶
In the case where static routes are not added, along with looking at CIS logs, see the below annotations to check if CNI is properly assigning podcidr and nodeip to the node.
Steps
- kubectl describe node
<nodename>. - Refer to the table below for annotations based on CNI configurations. CIS uses these annotations to read podCIDR and nodeIP allocated to the node to create route records dynamically on the BIG-IP.
| CNI Configured | Annotations/Spec Required | Description |
|---|---|---|
| ovn-k8s | OVNK8sNodeSubnetAnnotation = “k8s.ovn.org/node-subnets”,OVNK8sNodeIPAnnotation = “k8s.ovn.org/node-primary-ifaddr” by default or OVNK8sNodeIPAnnotation = “k8s.ovn.org/host-addresses” if –static-route-node-cidr is configured in CIS deployment args. | k8s.ovn.org/node-subnets is podCIDR allocated to the node. node-primary-ifaddr should have nodeip reachable from the BIG-IP |
| cilium-k8s | CiliumK8sNodeSubnetAnnotation12 = “io.cilium.network.ipv4-pod-cidr” or CiliumK8sNodeSubnetAnnotation13 = “network.cilium.io/ipv4-pod-cidr”, node ip from field node.Status.Addresses | io.cilium.network.ipv4-pod-cidr or network.cilium.io/ipv4-pod-cidr annotation is used based on cilium version to read podcidr. Nodeip is parsed from node manifest using field node.Status.Addresses. |
| antrea/flannel (default) | podcidr from node.Spec.PodCIDR, nodeIP from node.Status.Addresses | Podcidr is parsed from the node manifest using field node.Spec. PodCIDR and nodeip is parsed using the field node.Status.Addresses |