F5SPKVXLAN

The F5SPKVXLAN CR establishes a VXLAN tunnel between the Traffic Management Microkernel (TMM) and Kubernetes worker nodes. It configures the TMM to create one end-of the tunnel, specifying settings such as Virtual Network Identifier (VNI), Self IP addresses, and Maximum Transmission Unit (MTU). Meanwhile, the Calico Static Route Configuration (CSRC) established the other end of the tunnel on the Kubernetes worker nodes, ensuring a reliable, seamless and efficient data transmission pathway. For the full list of CRs, refer to the BIG-IP Next for Kubernetes CRs.

Parameters

The table below describe the F5SPKVXLAN CR spec parameters for F5SPKVXLAN configuration.

Parameter Description
name The name of the VXLAN object in the TMM configuration.
port Specifies the port to be used by UDP server to listen for VXLAN connections.
Note: Currently, the VXLAN port is always configured to 4789.
key Specifies the VNI to be used for the tunnel
remote_interface_name Specifies the interface on worker nodes on which VXLAN interface is created
remote_nodes.node_name Specifies the name of worker node on which VXLAN tunnel end point is created
remote_nodes.node_ip Specifies the IP address of the interface on the worker node (VXLAN remote endpoint for TMM), on which VXLAN tunnel is created
remote_nodes.peer_mac Specifies the MAC address of the VXLAN interface created on the worker node
remote_nodes.peerip_v4 Specifies the IPv4 address of the VXLAN interface created on the worker node
remote_nodes.peerip_v6 Specifies the IPv6 address of the VXLAN interface created on the worker node
local_ips Specifies the IP address of internal VLAN interface on the TMM (VXLAN local endpoint for TMM), on which VXLAN tunnel is created
selfip_v4s Specifies the IPv4 address of the VXLAN interface created on the each of the TMMs. Number of self_ipv4s should be equal to or more than number of TMMs.
prefixlen_v4 Specifies the prefix length for the IPv4 subnet assigned to the VXLAN interface created on the TMM
selfip_v6s Specifies the IPv6 address of the VXLAN interface created on the each of the TMMs. Number of self_ipv4s should be equal to or more than number of TMMs.
prefixlen_v6 Specifies the prefix length for the IPv6 subnet assigned to the VXLAN interface created on the TMM
mtu Specifies the MTU to be set for the VXLAN tunnel

F5SPKVXLAN CR Example

A sample setup of F5SPKVLAN CR and F5SPKVXLAN CR with two TMMs and three worker nodes.

apiVersion: "k8s.f5net.com/v1"
kind: F5SPKVlan
metadata:
  name: "internal"
spec:
  name: internal
  mtu: 8000
  tag: 0
  internal: true
  interfaces:
    - "1.2"
  selfip_v4s:
    [15.15.15.80]
  prefixlen_v4: 16
  selfip_v6s:
   ["fd00::15:15:15:80"]
  prefixlen_v6: 112
  auto_lasthop: "AUTO_LASTHOP_ENABLED"
Note: The IP addresses (20.0.0.201 and 20.0.0.202) reffered in the above F5VLAN example are referenced in the F5SPKVXLAN CR as a local_ips.
   apiVersion: "k8s.f5net.com/v1"
   kind: F5SPKVxlan
   metadata:
     name: "vxlan105"
   spec:
     name: "vxlan105"
     port: 4785
     key: 105
   remote_interface_name: "enP2s2f1v0" #internal ip interface on node
     remote_nodes:
       - node_name: "mgx-1" #node name
         node_ip: "15.15.15.51"  # node IP
         peer_mac: "1e:e4:4b:04:20:2f" # random mac
         peerip_v4: "10.19.0.50" # vxlan ip on node
   peerip_v6: "fd00::10:19:50:123" #vxlan ipv6 addr on node.
     local_ips:
       - "15.15.15.80" #TMM internal ip
     selfip_v4s:
       - "10.19.0.80" #vxlan ip address of tmm
     prefixlen_v4: 16
     selfip_v6s:
       - "fd00::10:19:50:124"
     prefixlen_v6: 112

Limitations:

  • Users must verify and configure the correct remote_interface_name
  • Users should ensure the correct node_name is configured under the remote_nodes field
  • Users need to configure the correct peer_mac address.
  • Users must configure the peerip_v4, peerip_v6, and self IP within the same subnet.
  • In the case of multiple VXLANs, users must configure unique peer IPs, self IPs, and peer_mac addresses.
  • Both peerip_v4 and peerip_v6 must be configured in a VXLAN CR for static route creation to occur.
  • Peer IPs and peer_mac configurations must be unique for each VXLAN.
  • VXLAN configuration updates are not supported; users must delete the existing VXLAN configuration and re-add it.
  • The TCP Offloading with VXLAN for Egress traffic is not supported on BIG-IP Next for Kubernetes v2.0.0-LA.

Installation

Requirements

To use the F5SPKVXLAN CR, ensure you have the following:

  • A Linux-based workstation.

Follow the instructions below to install the F5SPKVXLAN CR:

  1. Copy the below example F5SPKVXLAN CR to a YAML file, then modify the VXLAN settings as required and save:

    apiVersion: "k8s.f5net.com/v1"
    kind: F5SPKVXLAN
    metadata:
       name: "vxlan100"
    spec:
      name: "vxlan100"
      port: 4789
      key: 100
      remote_interface_name: "spk-ms"
      remote_nodes:
      - node_name: "datkube-control-plane"
        node_ip: "20.0.0.2"
        peer_mac: "00:f5:00:00:00:02"
        peerip_v4: "50.0.0.2"
        peerip_v6: "fd50::50:0:0:2"
      - node_name: "datkube-worker"
        node_ip: "20.0.0.3"
        peer_mac: "00:f5:00:00:00:03"
        peerip_v4: "50.0.0.3"
        peerip_v6: "fd50::50:0:0:3"
      - node_name: "datkube-worker2"
        node_ip: "20.0.0.4"
        peer_mac: "00:f5:00:00:00:04"
        peerip_v4: "50.0.0.4"
        peerip_v6: "fd50::50:0:0:4"  
      local_ips:
       - "20.0.0.201"
       - "20.0.0.202"
      selfip_v4s:
       - "50.0.0.201"
       - "50.0.0.202"
      prefixlen_v4: 24
      selfip_v6s:
       - "fd50::50:0:0:201"
       - "fd50::50:0:0:202"
      prefixlen_v6: 112
      mtu: 1460
    
  2. Install the F5SPKVXLAN CR that you have created:

    kubectl apply -f f5-spk-vxlan.yaml
    
  3. Verify the status of the installed F5SPKVXLAN CR:

    kubectl get f5-spk-vxlan
    
    NAME              READY    MESSAGE                                AGE
    vxlan100          TRUE     CR config sent to all grpc endpoints   27m 
    
  4. Verify the VXLAN tunnel endpoint in TMM:

    kubectl get pods -n default
    
     NAME                                          READY   STATUS    RESTARTS   AGE
     client                                        1/1     Running   0          33m
     dssm-f5-dssm-db-0                             2/2     Running   0          33m
     dssm-f5-dssm-db-1                             2/2     Running   0          32m
     dssm-f5-dssm-sentinel-0                       2/2     Running   0          33m
     dssm-f5-dssm-sentinel-1                       2/2     Running   0          32m
     f5-cert-manager-84f857f786-pbs5x              1/1     Running   0          34m
     f5-cert-manager-cainjector-695866d7ff-qz9pw   1/1     Running   0          34m
     f5-cert-manager-webhook-8554fd5b58-tll4s      1/1     Running   0          34m
     f5-crdconversion-84c68d6d5-zqj9f              1/1     Running   0          33m
     f5-rabbit-674b7cd47f-hht8t                    1/1     Running   0          34m
     f5-spk-csrc-4ppz8                             1/1     Running   0          31m
     f5-spk-csrc-jhvrz                             1/1     Running   0          31m
     f5-spk-csrc-qrpjc                             1/1     Running   0          31m
     f5-spk-cwc-689d5c9d97-vzp25                   2/2     Running   0          33m
     f5-tmm-6nwlj                                  4/4     Running   0          33m
     f5-tmm-hjd4c                                  4/4     Running   0          33m
     router                                        2/2     Running   0          33m
     spk-f5ingress-86559bbbb5-4tjsj                2/2     Running   0          33m
    
    kubectl exec -it f5-tmm-6nwlj -n default -c debug -- bash
    
    /ip a show dev vxlan100
    20: vxlan100: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
        link/ether 4a:b6:ed:70:26:4f brd ff:ff:ff:ff:ff:ff
        inet 50.0.0.201/24 brd 50.0.0.255 scope global vxlan100
           valid_lft forever preferred_lft forever
        inet6 fd50::50:0:0:201/112 scope global
           valid_lft forever preferred_lft forever
        inet6 fe80::d452:a8ff:fe77:1f9f/64 scope link
           valid_lft forever preferred_lft forever
        inet6 fe80::48b6:edff:fe70:264f/64 scope link
           valid_lft forever preferred_lft forever
    /
    
  5. Verify the VXLAN tunnel endpoint in Kubernetes worker node:

    kubectl exec -it datkube-worker ip a | grep vxlan100.100
    
     23: vxlan100.100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
         inet 50.0.0.3/32 scope global vxlan100.100