Top of Rack BGP¶
Overview¶
The Cloud-Native Network Functions (CNFs) Top of Rack (ToR) BGP feature enables the Traffic Management Microkernel (TMM) Pods to establish BGP peer relationships based on the cluster node the TMM Pod is running on. Without the ToR BGP feature, all TMM Pods must share the same BGP neighbor configuration, requiring a full BGP mesh.
Parameters¶
The table below describes the BIG-IP Controller’s BGP Helm parameters used in this document, refer to the BGP Overview for the full list of parameters.
Parameter | Description |
---|---|
asn |
The AS number of the f5-tmm-routing container. |
maxPathsEbgp |
Set the maxPathsEbgp to install multiple paths in External BGP |
maxPathsIbgp |
Set the maxPathsIbgp to install multiple paths in Internal BGP |
hostname |
The hostname of the f5-tmm-routing container. |
neighbors.nodeLookup |
Reference the ConfigMap holding the IP addresses. |
neighbors.asn |
The AS number of the BGP peer. |
neighbors.ebgpMultihop |
Enables connectivity between external peers that do not have a direct connection (1-255). |
neighbors.acceptsIPv4 |
Enables advertising IPv4 virtual server addresses to the peer (true / false). The default is false. |
neighbors.softReconf |
Enables BGP4 policies to be activated without clearing the BGP session. |
BGP peer groups¶
The CNFs ToR feature relies on BGP peer-groups to support per-node BGP peering relationships. Because BGP peer-groups support only a one AS (autonomous system) number, you must create a separate CNFs ConfigMap for each AS number when using the procedure below. You can use multiple ConfigMaps to map IP addresses within a single AS number, for example to map IPv4 and IPv6 peers within the AS
Procedure¶
Use these steps to configure CNFs ToR BGP peering.
Determine the cluster node names:
oc get nodes
NAME STATUS ROLES master-ocp.pd.f5net.com Ready control-plane,master worker1-ocp.pd.f5net.com Ready <none> worker2-ocp.pd.f5net.com Ready <none>
Create a new ConfigMap using the node names and the IP addresses of the BGP peers connected to the node. As mentioned in the BGP peer groups section, you must create one ConfgMap per AS number:
Note: Be certain to add the namespace of the BIG-IP Controller.
apiVersion: v1 kind: ConfigMap metadata: name: "cnf-tor-bgp" namespace: "cnf-gateway" data: worker1-ocp.f5.com: 10.20.2.206 worker2-ocp.f5.com: 10.20.2.207 master-ocp.f5.com: 10.20.2.208
You can also use multiple IP addresses per entry.
apiVersion: v1 kind: ConfigMap metadata: name: "cnf-tor-bgp" namespace: "cnf-gateway" data: worker1-ocp.f5.com: 10.20.2.106, 10.20.2.206 worker2-ocp.f5.com: 10.20.2.107, 10.20.2.207 master-ocp.f5.com: 10.20.2.108, 10.20.2.208
Reference the ConfigMap
name
using the BIG-IP Controller’sneighbors.nodeLookup
parameter:bgp: asn: 64443 maxPathsEbgp: 4 maxPathsIbgp: "null" hostname:cnf-gateway-bgp neighbors: - nodeLookup: cnf-tor-bgp asn: 3535 ebgpMultihop: 100 acceptsIPv4: true softReconf: true
After installing the BIG-IP Controller, to verify the BGP configuration is correct, first determine where the TMMs have scheduled:
oc get pods -n cnf-gateway -o wide | grep tmm
f5-tmm-f98ff99cc-l76hc 4/4 Running 172.18.0.5 worker1-ocp.f5.com f5-tmm-f98ff99cc-nfkgw 4/4 Running 172.18.0:6 worker2-ocp.f5.com
Log in to the f5-tmm-routing container:
In this example, the f5-tmm-routing container is in the cnf-gateway Project:
oc exec -it deploy/f5-tmm -c f5-tmm-routing -n cnf-gateway -- bash
Verify the correct IP address is configured:
imish
en
show running-config
In this example, the worker2-ocp.f5.com mapped IP address 10.20.2.207 is used in the configuration.
router bgp 64443 bgp router-id 0.0.107.127 no bgp default ipv4-unicast bgp log-neighbor-changes bgp graceful-restart restart-time 120 max-paths ebgp 4 redistribute kernel neighbor cnf-peers peer-group neighbor cnf-peers remote-as 3535 neighbor cnf-peers ebgp-multihop 100 neighbor cnf-peers activate neighbor cnf-peers soft-reconfiguration inbound neighbor 10.20.2.207 peer-group cnf-peers neighbor 10.20.2.207 activate
The TMM Pods should now peer with the BGP neighbors based on the node they are running.
Configuration updates¶
The configuration updates section describes how to update the CNFs BGP peering settings when either adding a new cluster node, or changing BGP peer IP addresses.
Adding new nodes¶
Use the steps below to add a new node to the cluster.
Note: These steps can also be followed for replacing nodes without interruption of service.
- Integrate the new cluster node.
- Configure the peer BGP Router for the new node.
- Update the CNFs BGP ConfigMap with the node and node IP address information.
- New TMM Pods can now be scheduled to the new node.
Changing BGP peer IPs¶
Use these step when changing BGP peer IP addresses.
Note: These steps can also be followed when BGP peers need to be replaced.
Configure the BGP peers with new IP addresses.
Update the CNFs ConfigMap by mapping each new BGP peer IP address with the correct cluster node name.
Delete the TMM Pods in the namespace:
oc delete pods -l app=f5-tmm -n cnf-gateway
The new TMM Pods should deploy automatically, reading the new BGP peer IP addresses from the CNFs config map.
Feedback¶
Provide feedback to improve this document by emailing cnfdocs@f5.com.