Cluster Requirements¶

Overview¶

Prior to integrating Cloud-Native Network Functions (CNFs) into the Robin Cloud-Native Platform (CNP), review this document to ensure the required software components are installed and properly configured.

Software support¶

The CNFs and Robin software versions listed below are the tested versions. F5 recommends these versions for the best performance and installation experience.

CNFs	Robin CNP
2.1.0	5.4.3-564
1.3.2 - 2.0.2	5.4.3-564
1.2.1 - 1.3.1	5.4.3-302
1.1.0 - 1.2.0	5.3.11-217
1.0.4 - 1.0.5	5.3.11-104
1.0.0	5.3.5-207

SR-IOV Support¶

Supported NICs¶

The following table lists the supported NICs tested on Robin Cloud Native Platform.

VF Information		PF Information				Robin Version
NICs	VF PCI IDs	PF PCI IDs	Kernel Network Driver	Network Drivers Version	Firmware	Robin Version
Intel XXV710	8086:154c	8086:158b	i40e	4.18.0-372.9.1.el8.x86_64	NVM 9.40	5.4.3-564 (HF5)
Intel E810-C	8086:1889	8086:1592	ice	1.17.2	4.0.0 4.40 0x8001c967 1.3534.0	5.4.3-564 (HF5)

Note: To use Advanced RSS on E810 NICs, your firmware must be version 4.0 or newer. Older firmware (3.10 or earlier) does not support this feature, and attempting to enable it will cause port initialization to fail, potentially disrupting your network. Update the firmware and drivers before enabling this feature to avoid network issues. Additionally, to ensure compatibility and proper operation of Advanced RSS, the PF ice driver must also be updated to the version 1.17.2.

Driver requirements¶

To ensure the BIG-IP Controller discovers and properly allocates Physical Functions (PF) to the TMM container, the following PF/VF driver requirements must be met:

E810 ice (SR-IOV PF) driver 1.17.2.
Enable SR-IOV support for E810 NICs.
Replace iavf with the vfio-pci VF driver in /etc/modprobe.d/vfio.conf:
```
blacklist iavf
options vfio-pci ids=8086:1889
```

Enable both IOMMU and hugepages in /etc/default/grub file:

GRUB_CMDLINE_LINUX should include iommu=pt intel_iommu=on hugepagesz=2M hugepages=32768  default_hugepagesz=2M vfio-pci.ids=8086:1889

Generate GRUB configuration file. The commands may differ if you are using BIOS or UEFI:

grub2-mkconfig -o /boot/grub2/grub.cfg

grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

Load the VFIO modules at boot time by adding the below entries to the /etc/modules-load.d/vfio.conf file:
```
vfio
vfio-pci
```
Configure VFs for SR-IOV capable NICs by adding the entries below to the /etc/rc.d/rc.local file:

Note: You may need to run chmod +x /etc/rc.d/rc.local.
```
echo 16 > /sys/class/net/p1p1/sriov_numvfs
echo 16 > /sys/class/net/p1p2/sriov_numvfs
```
Reboot the worker node.

Confirm hugepages are enabled on the worker node:

grep HugePages_Total /proc/meminfo

HugePages_Total:   32768

Confirm the VFIO modules have loaded:

lsmod | grep -i vfio

vfio_pci               41993  0
irqbypass              13503  2 kvm,vfio_pci
vfio_iommu_type1       22440  0
vfio                   32657  2 vfio_iommu_type1,vfio_pci

If the VF resources are not discovered by Kubernetes, use the robin host probe command to rediscover the host configuration:
```
robin host probe [<hostname>|--all] --rediscover
```

IP Pools¶

Configure the Robin IP Pools parameters with the following values:

Set the --spoofchk-disabled parameter.
Set the --trusted parameter.
Do not set the --vlan parameter, use the F5BigNetVlan CR tag parameter.

Example:

robin ip-pool add robin-pool-1095 --ranges 10.144.100.1-255 --prefix 16 --driver sriov \
--nictags name=p1p1 --vfdriver vfio-pci --trusted --spoofchk-disabled

CPU Allocation¶

Multiprocessor servers divide memory and CPUs into multiple NUMA nodes, each having a non-shared system bus. The CNF Controller requires that the CPUs and SR-IOV VFs allocated to the Traffic Management Microkernel (TMM) share the same NUMA node. To ensure the NUMA node alignment is handled properly, the Robin installation must include the following parameters and values:

The --cpu-manager-policy must be set to static.
The --topology-manager-policy must be set to single-numa-node.

Simultaneous Multithreading (SMT)¶

CNFs supports deployments in hyperthreading-enabled environments, enhancing scalability and resource utilization. This feature allows TMM to effectively manage logical CPUs, ensuring high performance in hyperthreaded setups.

For more information on managing this feature, see Simultaneous Multithreading and TMM Values sections.

Mitigations for CPU Resource Allocation¶

Following are a few mitigations and troubleshooting steps for addressing CPU resource allocation challenges in environments where hyperthreading (SMT) and Kubernetes static CPU policies are used.

Apply Intel microcode patches, Kernel/OS patches, or compiler mitigations such as “return trampoline” known as retpoline.
Prevent untrusted processes or deployments from sharing the same CPU; configure deployments to use two or more full physical cores.
Enable the full-pcpus-only policy with a static CPU Manager policy to ensure pods are allocated whole physical cores, avoiding partial allocations.
If the Kubernetes instance is configured properly with a static CPU policy and a full-pcpus-only policy, and the TMM starts with the correct count of CPU resources (For example 2), it will be assigned both threads of the same core, which means the whole core is assigned. No thread for the same core should be assigned to another work-load.
However, it is still dependent on how Kubernetes is configured, and how TMM starts. There should be two stages here.
1. When all are properly configured and the TMM has two threads of the same core, the mapres will detect these two threads as separate cores and will start two TMM threads. This initial mode should have a warning for performance implications.
2. Future implementation: When TMM starts with two threads, then the mapres (or similar) detects simultaneous multithreading, validates threads that belong to the same core, and only uses only one of these threads. This starts with only a single TMM thread per core by utilizing the entire core effectively.

Persistent storage¶

The required Fluentd logging collector, dSSM database and Traffic Management Microkernel (TMM) Debug Sidecar require an available Kubernetes persistent storage to bind to during installation.

Next step¶

Continue to the Getting Started guide to begin integrating CNFs.

Feedback¶

Provide feedback to improve this document by emailing cnfdocs@f5.com.

Supplemental¶

The CNI project.

Previous Next