Disaggregation (DAG) Modes¶

In F5 systems, DAG is a component responsible for distributing network traffic across multiple logical units. This page describes DAG modes for distribution between multiple Traffic Management Microkernel (TMM) cores within the TMM pod, as opposed to multiple TMM pods (see DAG CNFs).

Important: Choosing an inappropriate disaggregation mode can significantly degrade performance.

Default DAG¶

With Default DAG mode, the distribution of traffic is based on a source IP address, destination IP address, source port, and destination port of a packet. It provides the best source of entropy for the DAG and results in the most optimal distribution across TMM cores.

Service Provider DAG (SP-DAG)¶

The SP-DAG mode distributes packets based on client’s source IP on the external side and TMM’s self-IP on the internal side. The advantage of this mode is that all connections from the same client address always land on the same TMM core. To achieve proper load balancing, the CNFs system requires a sufficient range of self-IPs allocated on the internal side.

Key Features of SP-DAG¶

Following are some of the key features of Default DAG mode:

IP Address Based: Hashing is based entirely on source and destination IP addresses. For connections between TMM and clients, the client’s source IP address is used. For connections between TMM and pool members, the TMM’s self-IP address is used.
Used for a specific purpose: The main advantage of this mode is that a client with the same address always lands on the same TMM core.
Risk of Low Entropy: If TMM is assigned a small number of internal self-IPs, traffic may not be evenly load-balanced .

Software and Hardware DAG¶

The DAG inside TMM operates at two levels: software DAG and hardware DAG, each with a specific purpose to ensure optimal traffic management and load balancing.

Software DAG - Flexible Disaggregation¶

The software DAG operates within the TMM to handle traffic scenarios that cannot be processed entirely by the hardware DAG.
It supports advanced and customized disaggregation configurations required for application-specific traffic processing needs.
While more flexible, software DAG introduces additional overhead compared to hardware DAG.

Configuration¶

Software DAG is configured for each vlan with the option cmp_hash in the vlan CR. Posssible values:

SRC_DST_ADDR_PORT (default).
SRC_ADDR
DST_ADDR

Hardware DAG - High-Speed Traffic Processing¶

The hardware DAG is implemented on the Network Interface Card (NIC) and is designed for high-performance traffic distribution.
It hashes packets using attributes such as source/destination IP addresses and ports.
By handling disaggregation at the hardware level, it reduces dependency on CPU resources, enabling high-speed traffic processing.
Non-default DAG modes require specialzied hardware support.

Configuration¶

Hardware DAG is configured via tmm.customEnvVars parameters in the BIG-IP controller configuration. See BIG-IP controller reference. As opposed to the Software DAG, Hardware DAG is configured for an interface rather than a vlan.

The OPENSHIFT_VFIO_RESOURCE_<nbr> parameter is used to configure an interface See BIG-IP controller reference). The value for this parameter can be extended with an optional argument hw_hash_select that configures the hardware DAG mode for that interface.

The hw_hash_select argument takes a comma-separated list of tuples with no spaces in between. Available tuples are:

ip_src
ip_dst
port_src
port_dst

Examples:

- name: OPENSHIFT_VFIO_RESOURCE_1
- value: exampleResource1;hw_hash_select=ip_src

- name: OPENSHIFT_VFIO_RESOURCE_2
- value: exampleResource2;hw_hash_select=ip_dst

- name: OPENSHIFT_VFIO_RESOURCE_3
- value: exampleResource3;hw_hash_select=ip_src,ip_dst,port_src,port_dst

Aligning Software and Hardware DAGs with SP-DAG¶

To achieve optimal performance, it is imperative that the Software and Hardware DAGs are aligned. Failing to do so will result in packets being redirected between TMM cores on the software level, introducing substantial CPU overhead and degrading performance. For default DAG mode, no additional configuration is needed — Software and Hardware DAGs are aligned by default for every supported NIC.

Aligning DAGs in SP-DAG mode requires some consideration. Consider the following vlan configuration:

Vlan	Interfaces
external	1.1 and 1.2
internal	1.1 and 1.2

In this case, it is possible to configure Software DAG to use source IP for external, and destination IP for internal. Hardware DAG, however, cannot be aligned as interfaces 1.1 and 1.2 would have to use both source IP and destination IP depending on which vlan the packet belongs to.

Note: Aligning the DAGs is only possible when vlans with conflicting DAG modes do not share interfaces between each other.

Now, consider the following vlan configuration:

Vlan	Interfaces
external	1.1 and 1.2
internal	1.3 and 1.4

Aligning the DAGs in this case is possible:

Vlan	cmp_hash	Interfaces	OPENSHIFT_VFIO_RESOURCE_<nbr>
external	SRC_ADDR	1.1	exampleResource1;hw_hash_select=ip_src
		1.2	exampleResource2;hw_hash_select=ip_src
internal	DST_ADDR	1.3	exampleResource3;hw_hash_select=ip_dst
		1.4	exampleResource4;hw_hash_select=ip_dst

Hardware DAG support¶

	Intel E810	Intel 710 series	Mellanox CX-5	Mellanox CX-6	Mellanox CX-7
(default) ip_src,ip_dst, port_src,port_dst	+	+	+	+	+
ip_src,ip_dst	+	-	-	-	-
ip_src	+	-	-	-	-
ip_dst	+	-	-	-	-

Feedback

To provide feedback and help improve this document, please email us at cnfdocs@f5.com.

Previous Next