Last updated on: 2024-03-19 12:22:57.

StaticRouteSupport

Support for CIS to configure static routes in the BIG-IP with node subnets assigned to the nodes in the OpenShift/k8s cluster. This enables direct routing from BIG-IP to k8s Pods in cluster mode without vxaln tunnel configuration on BIG-IP.

Configuration

To enable the static route configuration, set --static-routing-mode to true and --orchestration-cni to CNI configured in the cluster.

args:
  --static-routing-mode=true
  --orchestration-cni=<ovn-k8s/flannel/antrea/cilium-k8s>

With CNI ovn-k8s, if the node has multiple interfaces, --static-route-node-cidr can be configured to specify the node network from which nodeip has to be selected. Without this config, CIS always picks the primary interface address from annotation k8s.ovn.org/node-primary-ifaddr on node manifest as nodeip for static route creation on the BIG-IP. Use cis-deployment-ovn-k8s-mnic.yaml to deploy with this configuration.

args:
 --static-route-node-cidr=10.4.0.0/14

Parameters for StaticRoutingMode

Parameter Type Required Default Description Allowed Values Agent Minimum Supported CIS Version
–static-routing-mode Boolean Optional false Adds Static Routes on the BIG-IP so traffic can be directly routed to the pods. (Without tunnels). true, false AS3 2.13.0
orchestration-cni String Optional flannel Kubernetes cluster CNI name. cilium-k8s, flannel, ovn-k8s, antrea AS3 2.13.0
shared-static-routes Boolean Optional false When set to true, static routes are created on the /Common partition, which can be valid only when static-routing-mode is enabled. true, false AS3 2.14.0
static-route-node- cidr String Optional N/A Specify the node network cidr to be used for static routing when the node has multiple interfaces. This is supported only with CNI ovn-k8s Any valid cidr eg: 10.4.0.0/14 AS3 2.15.0

cis-deployment-ovn-k8s.yaml

By deploying cis-deployment-ovn-k8s.yaml in your cluster, CIS will be configured with static route enabled for the ovn-k8s environment. The route will be configured with the name in the format k8s-<nodename>-<nodeip> as shown below.

BIG-IP Config

Validate the static routes added to the BIG-IP.

../_images/static-route.png

Note

  • StaticRoutingMode is required only with cluster mode where vxlan tunnel is not configured.
  • CIS uses –orchestration-cni to read node subnet info and nodeip based on the CNI configured.

Troubleshooting

In the case where static routes are not added, along with looking at CIS logs, see the below annotations to check if CNI is properly assigning podcidr and nodeip to the node.

Steps

  • kubectl describe node <nodename>.
  • Refer to the table below for annotations based on CNI configurations. CIS uses these annotations to read podCIDR and nodeIP allocated to the node to create route records dynamically on the BIG-IP.
CNI Configured Annotations/Spec Required Description
ovn-k8s OVNK8sNodeSubnetAnnotation = “k8s.ovn.org/node-subnets”,OVNK8sNodeIPAnnotation = “k8s.ovn.org/node-primary-ifaddr” by default or OVNK8sNodeIPAnnotation = “k8s.ovn.org/host-addresses” if –static-route-node-cidr is configured in CIS deployment args. k8s.ovn.org/node-subnets is podCIDR allocated to the node. node-primary-ifaddr should have nodeip reachable from the BIG-IP
cilium-k8s CiliumK8sNodeSubnetAnnotation12 = “io.cilium.network.ipv4-pod-cidr” or CiliumK8sNodeSubnetAnnotation13 = “network.cilium.io/ipv4-pod-cidr”, node ip from field node.Status.Addresses io.cilium.network.ipv4-pod-cidr or network.cilium.io/ipv4-pod-cidr annotation is used based on cilium version to read podcidr. Nodeip is parsed from node manifest using field node.Status.Addresses.
antrea/flannel (default) podcidr from node.Spec.PodCIDR, nodeIP from node.Status.Addresses Podcidr is parsed from the node manifest using field node.Spec. PodCIDR and nodeip is parsed using the field node.Status.Addresses