F5SPKVLAN

The F5SPKVLAN Custom Resource (CR) configures the Traffic Management Microkernel (TMM) network interface settings: VLAN tags, Self IP addresses, Maximum Transmission Size (MTU), bonding, and packet hashing algorithms. This overview discusses the F5SPKVLAN CR. For the full list of CRs, refer to the BIG-IP Next for Kubernetes CRs.

Scaling TMM

When scaling the TMM Pod beyond a single instance in the Project, the spec.selfip_v4s and spec.selfip_v6s parameters must be configured to provide unique self IP addresses to each TMM replica. The first self IP address in the list is applied to the first TMM Pod, the second IP address to the second TMM Pod, continuing through the list.

Annotate Internal facing interfaces

An annotation must be added to the Host node in order to establish static routes that will direct traffic to the TMM pod on the DPU. The IP’s CIDR range must match the “internal” network’s CIDR range.

kubectl get nodes -o yaml | grep node-primary-ifaddr

k8s.ovn.org/node-primary-ifaddr: '{"ipv4":"10.144.175.15/24","ipv6":"2620:128:e008:4018::15/128"}'

Note

If node node-primary-ifaddr annotation is not added already, use below command to annotate the node:

kubectl annotate node <host node name> k8s.ovn.org/node-primary-ifaddr='{"ipv4":"<IPv4 Address/subnet>","ipv6":"<IPv6 Address/subnet>"}'

Parameters

The CR spec parameters used to configure the BIG-IP Next for Kubernetes TMM network interfaces are:

Parameter

Description

Default value

name
string

The name of the VLAN object in the TMM configuration. The maximum length of the sting is 255.

“”

tag
integer

The tagging ID applied to the VLAN object. The minimum value is 0 and maximum is 4094.

0

bonded
boolean

Indicates the combine multiple interfaces into a single bonded interface.

false

interfaces
array

One or more interfaces associated with the VLAN object.

[]

internal
boolean

Indicates Routing annotations for internal Pods. This must be set on the internal VLAN, and can only be enabled on one VLAN.

false

selfip_v4s
array

Specifies a list of IPv4 Self IP addresses associated with the VLAN. Each TMM replica receives an IP address in the element order.

[]

prefixlen_v4
integer

The subnet mask used with IPv4 Self IP addresses. The minimum value is 0 and maximum is 32.

24

selfip_v6s
array

Specifies a list of IPv6 Self IP addresses associated with the VLAN. Each TMM replica receives an IP address in the element order.

[]

prefixlen_v6
integer

The subnet mask used with IPv6 Self IP addresses. The minimum value is 0 and maximum is 128.

120

mtu
integer

The maximum transmission unit in bytes that can be transmitted over a network connection. The minimum value is 576 and maximum is 9198.

1500

trunk_hash
string

The hashing algorithm used to distribute packets across bonded interfaces. Availabe options: src-dst-mac combines MAC addresses of the source and destination. dst-mac the MAC address of the destination. index combine ports of the source and the destination. src-dst-ipport combine IP addresses and ports of the source and the destination (default).

src-dst-ipport

auto_lasthop
string

Specifies the auto last hop feature that sends return traffic to the MAC address transmitting the request that can be AUTO_LASTHOP_DEFAULT, AUTO_LASTHOP_ENABLED or AUTO_LASTHOP_DISABLED.

“AUTO_LASTHOP_DEFAULT”

category
string

Specifies a unique, user-defined category for the VLAN, for example; serverside or clientside. The category value can then be referenced by the BIG-IP Next for Kubernetes CRs to either allow or deny VLAN traffic. The minimum value is 1 and maximum is 255.

“external”

allowed_services
array

Specifies a list of protocols and the protocol service ports the VLAN accepts.

[]

cmp_hash
string

Specifies the hashing algorithm to be used for the VLAN. Available options: DEFAULT, SRC_ADDR, DST_ADDR, SRC_DST_ADDR_PORT

“DEFAULT”

pod_hash
string

Specifies the pod hashing algorithm to be used for DAG. Available options: DISABLED, SRC_ADDR, DST_ADDR

“DISABLED”

allowed_services.items.protocol
string

The type of protocol traffic accepted by the VLAN. The maximum length of the sting is 32.

allowed_services.items.port
string

the service port traffic that can be accepted by the VLAN. The maximum length of the sting is 5.

“”

Requirements

Make sure you have:

  • Installed BNK (DPU).

Install an External and Internal F5SPKVLAN CR

Use the following steps to install an external and internal F5SPKVLAN CR, and verify the BIG-IP Next for Kubernetes TMM configuration.

  1. Copy the example CRs into a YAML file:

    Example external VLAN CR:

    apiVersion: "k8s.f5net.com/v1"
kind: F5SPKVlAN
metadata:
  namespace: f5-bnk
  name: "vlan-external"
spec:
  name: vlan-external
  tag: 3805
  bonded: true
  interfaces:
    - "1.1"
  selfip_v4s: 
    - "192.168.10.100"
    - "192.168.10.101"
    - "192.168.10.102"
  prefixlen_v4: 24
  selfip_v6s:
    - "aaaa::100"
    - "aaaa::101"
    - "aaaa::102"
  prefixlen_v6: 64
  mtu: 9000
  trunk_hash: src-dst-ipport
  auto_lasthop: "AUTO_LASTHOP_ENABLED"

    Example internal VLAN CR:

    apiVersion: "k8s.f5net.com/v1"
kind: F5SPKVLAN
metadata:
  namespace: f5-bnk
  name: "vlan-internal"
spec:
  name: vlan-internal
  tag: 3805
  internal: true
  interfaces:
    - "1.2"
  selfip_v4s: 
    - "10.144.175.100"
    - "10.144.175.101"
    - "10.144.175.102"
  prefixlen_v4: 24
  selfip_v6s:
    - "aaaa::100"
    - "aaaa::101"
    - "aaaa::102"
  prefixlen_v6: 64
  mtu: 9000
  trunk_hash: src-dst-ipport
  auto_lasthop: "AUTO_LASTHOP_DISABLED"

  2. Install the F5SPKVLAN CRs:

    kubectl apply -f spk-int-vlan.yaml

    kubectl apply -f spk-ext-vlan.yaml

  3. Verify the status of the installed CR:

    kubectl get f5-spk-vlans -n f5-bnk

    In this example, the CR has installed successfully.

    NAME            READY   MESSAGE                                          AGE

vlan-external True CR config sent to all grpc endpoints 2m20s vlan-internal True CR config sent to all grpc endpoints 2m20s


4. To verify the self IP address configuration, log in to the BIG-IP Next for Kubernetes TMM container:

_In this example, TMM is installed in the **f5-bnk** Project:_

```shell
kubectl exec -it deploy/f5-tmm -n f5-bnk -- bash

  1. List the interfaces and grep for the spec.name value:

    In this example, the VLAN spec.name is internal and the self IP address is 192.168.10.100:

    ip addr | grep -E 'internal|external'

    7:  vlan-external: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 
    inet 192.168.10.100/24 brd 10.20.0.0 scope global external
8:  vlan-internal: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000
    inet 10.144.175.100/24 brd 10.144.175.0 scope global internal

  2. TMM device assignments

    • Expected behaviour:

      • Device names are assigned to TMM pods based on smallest count between maxActiveReplicas and Least available selfIPs among all VLAN CRs.

      • TMM specific configurations other than VLAN such as snatpool, egress, and others can be configured on TMM pods only after VLAN configurations are applied.

      • If more than one TMM pods have device name assigned, all those TMM pods will have same number of VLAN interfaces (with unique selfIP) created even if selfIP count is different among the VLAN CRs.

      • TMM pods that are not assigned with a device name will remain in standby mode until any of the assigned TMM pod goes down or selfIP count is increased. TMMs on standby will continue receiving non-TMM specific configurations.

    • Active and Standby TMMs:

      • Active replicas are the TMM Pods that the controller sets up to handle traffic, but only if the CRs are set up to allow it.

      • When TMMs are scaled beyond the maxActiveReplicas, they will not be configured with self and snat IPs, even if such IPs are available for use. These additional TMMs are designated as Standbys TMMs, which means they are kept in a state of readiness to be quickly configured to take over the responsibilities of any Active TMM that may fail. This feature is particularly beneficial for the user who wish to minimize the duration of traffic interruption that can occur when a TMM Pod goes down. However, it is essential that users can allocate the necessary resources to support these additional TMMs. For example, if a user sets maxActiveReplicas to 4 and deploys a total of 6 TMMs, then 2 of those TMMs will be designated as standbys, ready to activate as needed.

      • If the minimum number of selfIPs across all VLANs is less than maxActiveReplicas, the effective maxActiveReplicas is limited to that minimum. For example, with ‘maxActiveReplicas’ set to 6, VLAN CR A having 6 self IPs, and VLAN CR B having 4 self IPs, the effective maxActiveReplicas is 4. Thus, only 4 TMMs can be configured for traffic processing.