F5SPKVLAN¶
The F5SPKVLAN Custom Resource (CR) configures the Traffic Management Microkernel (TMM) network interface settings: VLAN tags, Self IP addresses, Maximum Transmission Size (MTU), bonding, and packet hashing algorithms. This overview discusses the F5SPKVLAN CR. For the full list of CRs, refer to the BIG-IP Next for Kubernetes CRs.
Scaling TMM¶
When scaling the TMM Pod beyond a single instance in the Project, the spec.selfip_v4s
and spec.selfip_v6s
parameters must be configured to provide unique self IP addresses to each TMM replica. The first self IP address in the list is applied to the first TMM Pod, the second IP address to the second TMM Pod, continuing through the list.
Annotate Internal facing interfaces¶
An annotation must be added to the Host node in order to establish static routes that will direct traffic to the TMM pod on the DPU. The IP’s CIDR range must match the “internal” network’s CIDR range.
kubectl get nodes -o yaml | grep node-primary-ifaddr
k8s.ovn.org/node-primary-ifaddr: '{"ipv4":"10.144.175.15/24","ipv6":"2620:128:e008:4018::15/128"}'
Note: If nodenode-primary-ifaddr
annotation is not added already, use below command to annotate the node:
kubectl annotate node <host node name> k8s.ovn.org/node-primary-ifaddr='{"ipv4":"<IPv4 Address/subnet>","ipv6":"<IPv6 Address/subnet>"}'
Parameters¶
The CR spec
parameters used to configure the BIG-IP Next for Kubernetes TMM network interfaces are:
Parameter | Description |
---|---|
name |
The name of the VLAN object in the TMM configuration. |
tag |
The tagging ID applied to the VLAN object. Important: Do not set the OpenShift network attachment VLAN parameter, use the CR tag parameter. |
bonded |
Combine multiple interfaces into a single bonded interface (true/false). The default false (disabled). |
interfaces |
One or more interfaces to associate with the VLAN object. |
internal |
Enable Routing annotations for internal Pods (true/false). The default is false (disabled).This must be set on the internal VLAN, and can only be enabled on one VLAN. |
selfip_v4s |
Specifies a list of IPv4 Self IP addresses associated with the VLAN. Each TMM replica receives an IP address in the element order. |
prefixlen_v4 |
The IPv4 address subnet mask. |
selfip_v6s |
Specifies a list of IPv6 Self IP addresses associated with the VLAN. Each TMM replica receives an IP address in the element order. |
prefixlen_v6 |
The IPv6 address subnet mask. |
mtu |
Maximum transmission unit in bytes: (1500 to 9000). The default is 1500. Important: You must also set the BIG-IP Next for Kubernetes Controller TMM_DEFAULT_MTU parameter to the same value when modifying the default, and the value must be the same for each of the installed F5SPKVLAN CRs. |
trunk_hash |
The hashing algorithm used to distribute packets across bonded interfaces. Options: src-dst-mac combines MAC addresses of the source and destination. dst-mac the MAC address of the destination. index combine ports of the source and the destination. src-dst-ipport combine IP addresses and ports of the source and the destination (default). |
auto_lasthop |
Disables the auto last hop feature that sends return traffic to the MAC address transmitting the request: AUTO_LASTHOP_ENABLED, AUTO_LASTHOP_DISABLED or AUTO_LASTHOP_DEFAULT. |
category |
Specifies a unique, user-defined category for the VLAN, for example; serverside or clientside. The category value can then be referenced by the F5SPKIngressTCP, F5SPKIngressUDP, F5SPKIngressNGAP and F5SPKIngressGTP BIG-IP Next for Kubernetes CRs to either allow or deny VLAN traffic. |
allowed_services |
Specifies a list of protocols and the protocol service ports this VLAN accepts. |
allowed_services.protocol |
Specifies the protocol traffic the VLAN accepts. |
allowed_services.port |
Specifies the service port traffic the VLAN accepts. |
Install an External and Internal F5SPKVLAN CR¶
Use the following steps to install an external and internal F5SPKVLAN CR, and verify the BIG-IP Next for Kubernetes TMM configuration.
Copy the example CRs into a YAML file:
Example external VLAN CR:
apiVersion: "k8s.f5net.com/v1" kind: F5SPKVLAN metadata: namespace: default name: "vlan-external" spec: name: external tag: 3805 bonded: true interfaces: - "1.1" - "1.2" selfip_v4s: - "192.168.10.100" - "192.168.10.101" - "192.168.10.102" prefixlen_v4: 24 selfip_v6s: - "aaaa::100" - "aaaa::101" - "aaaa::102" prefixlen_v6: 64 mtu: 9000 trunk_hash: src-dst-ipport auto_lasthop: "AUTO_LASTHOP_ENABLED"
Example internal VLAN CR:
apiVersion: "k8s.f5net.com/v1" kind: F5SPKVLAN metadata: namespace: default name: "vlan-internal" spec: name: internal tag: 3805 internal: true interfaces: - "1.3" - "1.4" selfip_v4s: - "10.144.175.100" - "10.144.175.101" - "10.144.175.102" prefixlen_v4: 24 selfip_v6s: - "aaaa::100" - "aaaa::101" - "aaaa::102" prefixlen_v6: 64 mtu: 9000 trunk_hash: src-dst-ipport auto_lasthop: "AUTO_LASTHOP_DISABLED"
Install the F5SPKVLAN CRs:
kubectl apply -f spk-int-vlan.yaml
kubectl apply -f spk-ext-vlan.yaml
Verify the status of the installed CR:
kubectl get f5-spk-vlan -n default
In this example, the CR has installed successfully.
NAME STATUS MESSAGE staticroute-ipv4 SUCCESS CR config sent to all grpc endpoints
To verify the self IP address configuration, log in to the BIG-IP Next for Kubernetes TMM container:
In this example, TMM is installed in the default Project:
kubectl exec -it deploy/f5-tmm -n default -- bash
List the interfaces and grep for the
spec.name
value:In this example, the VLAN
spec.name
is internal and the self IP address is 192.168.10.100:ip addr | grep -E 'internal|external'
7: external: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 inet 192.168.10.100/24 brd 10.20.0.0 scope global external 8: internal: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 inet 10.144.175.100/24 brd 10.144.175.0 scope global internal
Note: With multiple VLAN CRs,
selfIP
from first index in all CRs will be assigned to same TMM. This TMM will have device name asDP-0
.SelfIPs
from second index in all CRs will be assigned to same TMM, and has the device name asDP-1
and so on.TMM device assignments
- Expected behaviour:
- Device names are assigned to TMM pods based on smallest count between
maxActiveReplicas
and Least availableselfIPs
among all VLAN CRs. - TMM specific configurations other than VLAN such as snatpool, egress, and others can be configured on TMM pods only after VLAN configurations are applied.
- If more than one TMM pods have device name assigned, all those TMM pods will have same number of VLAN interfaces (with unique
selfIP
) created even ifselfIP
count is different among the VLAN CRs. - TMM pods that are not assigned with a device name will remain in standby mode until any of the assigned TMM pod goes down or
selfIP
count is increased. TMMs on standby will continue receiving non-TMM specific configurations.
- Device names are assigned to TMM pods based on smallest count between
- Active and Standby TMMs:
- Active replicas are the TMM Pods that the controller sets up to handle traffic, but only if the CRs are set up to allow it.
- When TMMs are scaled beyond the
maxActiveReplicas
, they will not be configured with self and snat IPs, even if such IPs are available for use. These additional TMMs are designated as Standbys TMMs, which means they are kept in a state of readiness to be quickly configured to take over the responsibilities of any Active TMM that may fail. This feature is particularly beneficial for the user who wish to minimize the duration of traffic interruption that can occur when a TMM Pod goes down. However, it is essential that users can allocate the necessary resources to support these additional TMMs. For example, if a user setsmaxActiveReplicas
to 4 and deploys a total of 6 TMMs, then 2 of those TMMs will be designated as standbys, ready to activate as needed. - If the minimum number of selfIPs across all VLANs is less than
maxActiveReplicas
, the effectivemaxActiveReplicas
is limited to that minimum. For example, with ‘maxActiveReplicas’ set to 6, VLAN CR A having 6 self IPs, and VLAN CR B having 4 self IPs, the effectivemaxActiveReplicas
is 4. Thus, only 4 TMMs can be configured for traffic processing.
- Expected behaviour: