F5SPKVXLAN¶
The F5SPKVXLAN CR establishes a VXLAN tunnel between the Traffic Management Microkernel (TMM) and Kubernetes worker nodes. It configures the TMM to create one end-of the tunnel, specifying settings such as Virtual Network Identifier (VNI), Self IP addresses, and Maximum Transmission Unit (MTU). Meanwhile, the Calico Static Route Configuration (CSRC) established the other end of the tunnel on the Kubernetes worker nodes, ensuring a reliable, seamless and efficient data transmission pathway. For the full list of CRs, refer to the BIG-IP Next for Kubernetes CRs.
Parameters¶
The table below describe the F5SPKVXLAN CR spec
parameters for F5SPKVXLAN configuration.
Parameter | Description |
---|---|
name |
The name of the VXLAN object in the TMM configuration. |
port |
Specifies the port to be used by UDP server to listen for VXLAN connections. |
Note: Currently, the VXLAN port is always configured to 4789. | |
key |
Specifies the VNI to be used for the tunnel |
remote_interface_name |
Specifies the interface on worker nodes on which VXLAN interface is created |
remote_nodes.node_name |
Specifies the name of worker node on which VXLAN tunnel end point is created |
remote_nodes.node_ip |
Specifies the IP address of the interface on the worker node (VXLAN remote endpoint for TMM), on which VXLAN tunnel is created |
remote_nodes.peer_mac |
Specifies the MAC address of the VXLAN interface created on the worker node |
remote_nodes.peerip_v4 |
Specifies the IPv4 address of the VXLAN interface created on the worker node |
remote_nodes.peerip_v6 |
Specifies the IPv6 address of the VXLAN interface created on the worker node |
local_ips |
Specifies the IP address of internal VLAN interface on the TMM (VXLAN local endpoint for TMM), on which VXLAN tunnel is created |
selfip_v4s |
Specifies the IPv4 address of the VXLAN interface created on the each of the TMMs. Number of self_ipv4s should be equal to or more than number of TMMs. |
prefixlen_v4 |
Specifies the prefix length for the IPv4 subnet assigned to the VXLAN interface created on the TMM |
selfip_v6s |
Specifies the IPv6 address of the VXLAN interface created on the each of the TMMs. Number of self_ipv4s should be equal to or more than number of TMMs. |
prefixlen_v6 |
Specifies the prefix length for the IPv6 subnet assigned to the VXLAN interface created on the TMM |
mtu |
Specifies the MTU to be set for the VXLAN tunnel |
F5SPKVXLAN CR Example¶
A sample setup of F5SPKVLAN CR and F5SPKVXLAN CR with two TMMs and three worker nodes.
apiVersion: "k8s.f5net.com/v1"
kind: F5SPKVlan
metadata:
name: "internal"
spec:
name: internal
mtu: 8000
tag: 0
internal: true
interfaces:
- "1.2"
selfip_v4s:
[15.15.15.80]
prefixlen_v4: 16
selfip_v6s:
["fd00::15:15:15:80"]
prefixlen_v6: 112
auto_lasthop: "AUTO_LASTHOP_ENABLED"
Note: The IP addresses (20.0.0.201 and 20.0.0.202) reffered in the above F5VLAN example are referenced in the F5SPKVXLAN CR as a local_ips.
apiVersion: "k8s.f5net.com/v1"
kind: F5SPKVxlan
metadata:
name: "vxlan105"
spec:
name: "vxlan105"
port: 4785
key: 105
remote_interface_name: "enP2s2f1v0" #internal ip interface on node
remote_nodes:
- node_name: "mgx-1" #node name
node_ip: "15.15.15.51" # node IP
peer_mac: "1e:e4:4b:04:20:2f" # random mac
peerip_v4: "10.19.0.50" # vxlan ip on node
peerip_v6: "fd00::10:19:50:123" #vxlan ipv6 addr on node.
local_ips:
- "15.15.15.80" #TMM internal ip
selfip_v4s:
- "10.19.0.80" #vxlan ip address of tmm
prefixlen_v4: 16
selfip_v6s:
- "fd00::10:19:50:124"
prefixlen_v6: 112
Limitations:¶
- Users must verify and configure the correct
remote_interface_name
- Users should ensure the correct node_name is configured under the
remote_nodes field
- Users need to configure the correct
peer_mac
address. - Users must configure the peerip_v4, peerip_v6, and self IP within the same subnet.
- In the case of multiple VXLANs, users must configure unique peer IPs, self IPs, and peer_mac addresses.
- Both
peerip_v4
andpeerip_v6
must be configured in a VXLAN CR for static route creation to occur. - Peer IPs and
peer_mac
configurations must be unique for each VXLAN. - VXLAN configuration updates are not supported; users must delete the existing VXLAN configuration and re-add it.
- The TCP Offloading with VXLAN for Egress traffic is not supported on BIG-IP Next for Kubernetes v2.0.0-LA.
Installation¶
Requirements¶
To use the F5SPKVXLAN CR, ensure you have the following:
- A Linux-based workstation.
Follow the instructions below to install the F5SPKVXLAN CR:
Copy the below example F5SPKVXLAN CR to a YAML file, then modify the VXLAN settings as required and save:
apiVersion: "k8s.f5net.com/v1" kind: F5SPKVXLAN metadata: name: "vxlan100" spec: name: "vxlan100" port: 4789 key: 100 remote_interface_name: "spk-ms" remote_nodes: - node_name: "datkube-control-plane" node_ip: "20.0.0.2" peer_mac: "00:f5:00:00:00:02" peerip_v4: "50.0.0.2" peerip_v6: "fd50::50:0:0:2" - node_name: "datkube-worker" node_ip: "20.0.0.3" peer_mac: "00:f5:00:00:00:03" peerip_v4: "50.0.0.3" peerip_v6: "fd50::50:0:0:3" - node_name: "datkube-worker2" node_ip: "20.0.0.4" peer_mac: "00:f5:00:00:00:04" peerip_v4: "50.0.0.4" peerip_v6: "fd50::50:0:0:4" local_ips: - "20.0.0.201" - "20.0.0.202" selfip_v4s: - "50.0.0.201" - "50.0.0.202" prefixlen_v4: 24 selfip_v6s: - "fd50::50:0:0:201" - "fd50::50:0:0:202" prefixlen_v6: 112 mtu: 1460
Install the F5SPKVXLAN CR that you have created:
kubectl apply -f f5-spk-vxlan.yaml
Verify the status of the installed F5SPKVXLAN CR:
kubectl get f5-spk-vxlan
NAME READY MESSAGE AGE vxlan100 TRUE CR config sent to all grpc endpoints 27m
Verify the VXLAN tunnel endpoint in TMM:
kubectl get pods -n default
NAME READY STATUS RESTARTS AGE client 1/1 Running 0 33m dssm-f5-dssm-db-0 2/2 Running 0 33m dssm-f5-dssm-db-1 2/2 Running 0 32m dssm-f5-dssm-sentinel-0 2/2 Running 0 33m dssm-f5-dssm-sentinel-1 2/2 Running 0 32m f5-cert-manager-84f857f786-pbs5x 1/1 Running 0 34m f5-cert-manager-cainjector-695866d7ff-qz9pw 1/1 Running 0 34m f5-cert-manager-webhook-8554fd5b58-tll4s 1/1 Running 0 34m f5-crdconversion-84c68d6d5-zqj9f 1/1 Running 0 33m f5-rabbit-674b7cd47f-hht8t 1/1 Running 0 34m f5-spk-csrc-4ppz8 1/1 Running 0 31m f5-spk-csrc-jhvrz 1/1 Running 0 31m f5-spk-csrc-qrpjc 1/1 Running 0 31m f5-spk-cwc-689d5c9d97-vzp25 2/2 Running 0 33m f5-tmm-6nwlj 4/4 Running 0 33m f5-tmm-hjd4c 4/4 Running 0 33m router 2/2 Running 0 33m spk-f5ingress-86559bbbb5-4tjsj 2/2 Running 0 33m
kubectl exec -it f5-tmm-6nwlj -n default -c debug -- bash
/ip a show dev vxlan100 20: vxlan100: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000 link/ether 4a:b6:ed:70:26:4f brd ff:ff:ff:ff:ff:ff inet 50.0.0.201/24 brd 50.0.0.255 scope global vxlan100 valid_lft forever preferred_lft forever inet6 fd50::50:0:0:201/112 scope global valid_lft forever preferred_lft forever inet6 fe80::d452:a8ff:fe77:1f9f/64 scope link valid_lft forever preferred_lft forever inet6 fe80::48b6:edff:fe70:264f/64 scope link valid_lft forever preferred_lft forever /
Verify the VXLAN tunnel endpoint in Kubernetes worker node:
kubectl exec -it datkube-worker ip a | grep vxlan100.100
23: vxlan100.100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default inet 50.0.0.3/32 scope global vxlan100.100