Top of Rack BGP

Overview

The Cloud-Native Network Functions (CNFs) Top of Rack (ToR) BGP feature enables the Traffic Management Microkernel (TMM) Pods to establish BGP peer relationships based on the cluster node the TMM Pod is running on. Without the ToR BGP feature, all TMM Pods must share the same BGP neighbor configuration, requiring a full BGP mesh.

Parameters

The table below describes the BIG-IP Controller’s BGP Helm parameters used in this document, refer to the BGP Overview for the full list of parameters.

Parameter Description
asn The AS number of the f5-tmm-routing container.
maxPathsEbgp Set the maxPathsEbgp to install multiple paths in External BGP
maxPathsIbgp Set the maxPathsIbgp to install multiple paths in Internal BGP
hostname The hostname of the f5-tmm-routing container.
neighbors.nodeLookup Reference the ConfigMap holding the IP addresses.
neighbors.asn The AS number of the BGP peer.
neighbors.ebgpMultihop Enables connectivity between external peers that do not have a direct connection (1-255).
neighbors.acceptsIPv4 Enables advertising IPv4 virtual server addresses to the peer (true / false). The default is false.
neighbors.softReconf Enables BGP4 policies to be activated without clearing the BGP session.

BGP peer groups

The CNFs ToR feature relies on BGP peer-groups to support per-node BGP peering relationships. Because BGP peer-groups support only a one AS (autonomous system) number, you must create a separate CNFs ConfigMap for each AS number when using the procedure below. You can use multiple ConfigMaps to map IP addresses within a single AS number, for example to map IPv4 and IPv6 peers within the AS

Procedure

Use these steps to configure CNFs ToR BGP peering.

  1. Determine the cluster node names:

    oc get nodes
    
    NAME                          STATUS   ROLES                 
    master-ocp.pd.f5net.com    Ready    control-plane,master   
    worker1-ocp.pd.f5net.com   Ready    <none>                 
    worker2-ocp.pd.f5net.com   Ready    <none>   
    
  2. Create a new ConfigMap using the node names and the IP addresses of the BGP peers connected to the node. As mentioned in the BGP peer groups section, you must create one ConfgMap per AS number:

    Note: Be certain to add the namespace of the BIG-IP Controller.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: "cnf-tor-bgp"
      namespace: "cnf-gateway"
    data:
      worker1-ocp.f5.com: 10.20.2.206
      worker2-ocp.f5.com: 10.20.2.207
      master-ocp.f5.com: 10.20.2.208
    

    You can also use multiple IP addresses per entry.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: "cnf-tor-bgp"
      namespace: "cnf-gateway"
    data:
      worker1-ocp.f5.com: 10.20.2.106, 10.20.2.206
      worker2-ocp.f5.com: 10.20.2.107, 10.20.2.207
      master-ocp.f5.com: 10.20.2.108, 10.20.2.208
    
  3. Reference the ConfigMap name using the BIG-IP Controller’s neighbors.nodeLookup parameter:

    bgp:
      asn: 64443
      maxPathsEbgp: 4
      maxPathsIbgp: "null"
      hostname:cnf-gateway-bgp
      neighbors:
       - nodeLookup: cnf-tor-bgp
         asn: 3535
         ebgpMultihop: 100
         acceptsIPv4: true
         softReconf: true            
    
  4. After installing the BIG-IP Controller, to verify the BGP configuration is correct, first determine where the TMMs have scheduled:

    oc get pods -n cnf-gateway -o wide | grep tmm
    
    f5-tmm-f98ff99cc-l76hc  4/4  Running  172.18.0.5  worker1-ocp.f5.com
    f5-tmm-f98ff99cc-nfkgw  4/4  Running  172.18.0:6  worker2-ocp.f5.com
    
  5. Log in to the f5-tmm-routing container:

    In this example, the f5-tmm-routing container is in the cnf-gateway Project:

    oc exec -it deploy/f5-tmm -c f5-tmm-routing -n cnf-gateway -- bash
    
  6. Verify the correct IP address is configured:

    imish
    
    en
    
    show running-config
    

    In this example, the worker2-ocp.f5.com mapped IP address 10.20.2.207 is used in the configuration.

    router bgp 64443
    bgp router-id 0.0.107.127
    no bgp default ipv4-unicast
    bgp log-neighbor-changes
    bgp graceful-restart restart-time 120
    max-paths ebgp 4
    redistribute kernel
    neighbor cnf-peers peer-group
    neighbor cnf-peers remote-as 3535
    neighbor cnf-peers ebgp-multihop 100
    neighbor cnf-peers activate
    neighbor cnf-peers soft-reconfiguration inbound
    neighbor 10.20.2.207 peer-group cnf-peers
    neighbor 10.20.2.207 activate
    
  7. The TMM Pods should now peer with the BGP neighbors based on the node they are running.

Configuration updates

The configuration updates section describes how to update the CNFs BGP peering settings when either adding a new cluster node, or changing BGP peer IP addresses.

Adding new nodes

Use the steps below to add a new node to the cluster.

Note: These steps can also be followed for replacing nodes without interruption of service.

  1. Integrate the new cluster node.
  2. Configure the peer BGP Router for the new node.
  3. Update the CNFs BGP ConfigMap with the node and node IP address information.
  4. New TMM Pods can now be scheduled to the new node.

Changing BGP peer IPs

Use these step when changing BGP peer IP addresses.

Note: These steps can also be followed when BGP peers need to be replaced.

  1. Configure the BGP peers with new IP addresses.

  2. Update the CNFs ConfigMap by mapping each new BGP peer IP address with the correct cluster node name.

  3. Delete the TMM Pods in the namespace:

    oc delete pods -l app=f5-tmm -n cnf-gateway
    
  4. The new TMM Pods should deploy automatically, reading the new BGP peer IP addresses from the CNFs config map.

Feedback

Provide feedback to improve this document by emailing cnfdocs@f5.com.