Top of Rack BGP¶

Overview¶

The Service Proxy for Kubernetes (SPK) Top of Rack (ToR) BGP feature enables the Traffic Management Microkernel (TMM) Pods to establish BGP peer relationships based on the cluster node the TMM Pod is running on. Without the ToR BGP feature, all TMM Pods must share the same BGP neighbor configuration, requiring a full BGP mesh.

Parameters¶

The table below describes the BIG-IP Controller’s BGP Helm parameters used in this document, refer to the BGP Overview for the full list of parameters.

Parameter	Description
`asn`	The AS number of the f5-tmm-routing container.
`maxPathsEbgp`	Set the maxPathsEbgp to install multiple paths in External BGP
`maxPathsIbgp`	Set the maxPathsIbgp to install multiple paths in Internal BGP
`hostname`	The hostname of the f5-tmm-routing container.
`neighbors.nodeLookup`	Reference the ConfigMap holding the IP addresses.
`neighbors.asn`	The AS number of the BGP peer.
`neighbors.ebgpMultihop`	Enables connectivity between external peers that do not have a direct connection (1-255).
`neighbors.acceptsIPv4`	Enables advertising IPv4 virtual server addresses to the peer (true / false). The default is false.
`neighbors.softReconf`	Enables BGP4 policies to be activated without clearing the BGP session.

BGP peer groups¶

The SPKs ToR feature relies on BGP peer-groups to support per-node BGP peering relationships. Because BGP peer-groups support only a one AS (autonomous system) number, you must create a separate SPKs ConfigMap for each AS number when using the procedure below. You can use multiple ConfigMaps to map IP addresses within a single AS number, for example to map IPv4 and IPv6 peers within the AS

Procedure¶

Use these steps to configure SPKs ToR BGP peering.

Determine the cluster node names:

kubectl get nodes

NAME                          STATUS   ROLES                 
master-robin3.pd.f5net.com    Ready    control-plane,master   
worker1-robin3.pd.f5net.com   Ready    <none>                 
worker2-robin3.pd.f5net.com   Ready    <none>   

Create a new ConfigMap using the node names and the IP addresses of the BGP peers connected to the node. As mentioned in the BGP peer groups section, you must create one ConfgMap per AS number:

Note: Be certain to add the namespace of the BIG-IP Controller.

apiVersion: v1
kind: ConfigMap
metadata:
  name: "spk-tor-bgp"
  namespace: "spk-ingress"
data:
  worker1-robin.f5.com: 10.20.2.206
  worker2-robin.f5.com: 10.20.2.207
  master-robin.f5.com: 10.20.2.208

You can also use multiple IP addresses per entry.

apiVersion: v1
kind: ConfigMap
metadata:
  name: "spk-tor-bgp"
  namespace: "spk-ingress"
data:
  worker1-robin.f5.com: 10.20.2.106, 10.20.2.206
  worker2-robin.f5.com: 10.20.2.107, 10.20.2.207
  master-robin.f5.com: 10.20.2.108, 10.20.2.208

Reference the ConfigMap name using the BIG-IP Controller’s neighbors.nodeLookup parameter:

bgp:
  asn: 64443
  maxPathsEbgp: 4
  maxPathsIbgp: "null"
  hostname:spk-ingress-bgp
  neighbors:
   - nodeLookup: spk-tor-bgp
     asn: 3535
     ebgpMultihop: 100
     acceptsIPv4: true
     softReconf: true            

After installing the BIG-IP Controller, to verify the BGP configuration is correct, first determine where the TMMs have scheduled:

kubectl get pods -n spk-ingress -o wide | grep tmm

f5-tmm-f98ff99cc-l76hc  4/4  Running  172.18.0.5  worker1-robin.f5.com
f5-tmm-f98ff99cc-nfkgw  4/4  Running  172.18.0:6  worker2-robin.f5.com

View the ZebOS configuration once the BIG-IP Controller has installed:

In this example, the f5-tmm-routing container is in the spk-ingress Project:
```
oc exec -it deploy/f5-tmm -c f5-tmm-routing -n spk-ingress \
-- imish -e 'show running-config'
```

Verify the correct IP address is configured:

In this example, the worker2-robin.f5.com mapped IP address 10.20.2.207 is used in the configuration.

router bgp 64443
bgp router-id 0.0.107.127
no bgp default ipv4-unicast
bgp log-neighbor-changes
bgp graceful-restart restart-time 120
max-paths ebgp 4
redistribute kernel
neighbor spk-peers peer-group
neighbor spk-peers remote-as 3535
neighbor spk-peers ebgp-multihop 100
neighbor spk-peers activate
neighbor spk-peers soft-reconfiguration inbound
neighbor 10.20.2.207 peer-group spk-peers
neighbor 10.20.2.207 activate

The TMM Pods should now peer with the BGP neighbors based on the node they are running.

Configuration updates¶

The configuration updates section describes how to update the SPKs BGP peering settings when either adding a new cluster node, or changing BGP peer IP addresses.

Adding new nodes¶

Use the steps below to add a new node to the cluster.

Note: These steps can also be followed for replacing nodes without interruption of service.

Integrate the new cluster node.
Configure the peer BGP Router for the new node.
Update the SPKs BGP ConfigMap with the node and node IP address information.
New TMM Pods can now be scheduled to the new node.

Changing BGP peer IPs¶

Use these step when changing BGP peer IP addresses.

Note: These steps can also be followed when BGP peers need to be replaced.

Configure the BGP peers with new IP addresses.
Update the SPKs ConfigMap by mapping each new BGP peer IP address with the correct cluster node name.

Delete the TMM Pods in the namespace:

kubectl delete pods -l app=f5-tmm -n spk-ingress

The new TMM Pods should deploy automatically, reading the new BGP peer IP addresses from the SPKs config map.

Feedback¶

Provide feedback to improve this document by emailing spkdocs@f5.com.

Previous Next