Cluster Requirements¶

Overview¶

Prior to integrating Service Proxy for Kubernetes (SPK) into the OpenShift cluster, review this document to ensure the required software components are installed and properly configured.

Software support¶

The SPK and Red Hat software versions listed below are the tested versions. F5 recommends these versions for the best performance and installation experience.

SPK	OpenShift
2.1.0	4.14.42 and 4.16.37
2.0.0 - 2.0.2	4.14.42 and 4.16.37
1.9.2	4.14.26
1.9.1	4.14.10
1.9.0	4.12.26 and 4.14.1
1.8.2	4.12.26
1.8.0	4.12.22
1.7.14	4.12.53 and 4.14.42
1.7.10 - 1.7.11	4.12.59
1.7.5 - 1.7.9	4.12.45
1.7.4	4.12.44
1.7.3	4.12.26
1.7.2	4.12.14
1.7.1	4.12.7
1.7.0	4.12.1
1.5.0 - 1.6.1	4.10.13
1.4.12 - 1.4.15	4.8.10
1.3.1 - 1.4.11	4.7.8

Pod Networking¶

To support low-latency 5G workloads, SPK relies on Single Root I/O Virtualization (SR-IOV) and the Open Virtual Network with Kubernetes (OVN-Kubernetes) CNI. SPK supports the Kubernetes version based in vanilla Kubernetes 1.x and CNI such as Calico, OpenShfit´s OVN.

To ensure the cluster supports multi-homed Pods; the ability to select either the default (virtual) CNI or the SR-IOV / OVN-Kubernetes (physical) CNI, review the sections below.

Network Operator¶

To properly manage the cluster networks, the OpenShift Cluster Network Operator must be installed.

Important: OpenShift 4.8 requires configuring local gateway mode using the steps below:

Create the manifest files:

openshift-install --dir=<install dir> create cluster

Create a ConfigMap in new manifest directory, and add the following YAML code:

apiVersion: v1
kind: ConfigMap
metadata:
    name: gateway-mode-config
    namespace: openshift-network-operator
data:
    mode: "local"
immutable: true

Create the cluster:

openshift-install create cluster --dir=<install dir>

The Cluster Network Operator installation on Github.

SR-IOV¶

Supported NICs¶

The table below lists the currently supported NICs.

NICs	VF Driver	VF PCI ID	Firmware
Intel XXV710	140e	8086:154c	1.3429.0 (NVM 9.4)
Mellanox ConnectX-5	mlx5_core	15b3:1018	16.35.3006
Mellanox ConnectX-6 Lx	mlx5_core	15b3:101e	26.41.1000
Mellanox ConnectX-6 Dx	mlx5_core	15b3:101e	22.41.1000

VF Configuration¶

To define the SR-IOV Virtual Functions (VFs) used by the Service Proxy Traffic Management Microkernel (TMM), configure the following OpenShift network objects:

An external and internal Network node policy.
An external and internal Network attachment definition.
- Set the spoofChk parameter to off.
- Set the trust parameter to on.
- Set the capabilities parameter to '{"mac": true, "ips": true}'.
- Do not set the vlan parameter, set the F5SPKVlan tag parameter.
- Do not set the ipam parameter, set the F5SPKVlan internal parameter.

Refer to the SPK Config File Reference for examples.

CPU Allocation¶

Multiprocessor servers divide memory and CPUs into multiple NUMA nodes, each having a non-shared system bus. When installing the SPK Controller, the CPUs and SR-IOV VFs allocated to the Service Proxy TMM container must share the same NUMA node. To ensure the CPU NUMA node alignment is handled properly by the cluster ensure the following parameters are set:

Set the Performance Profile’s Topology Manager Policy to single-numa-node.
Set the Kubelet configuration’s CPU Manager Policy to static in the Kubelet configuration.

Simultaneous Multithreading (SMT)¶

CNFs supports deployments in hyperthreading-enabled environments, enhancing scalability and resource utilization. This feature allows TMM to effectively manage logical CPUs, ensuring high performance in hyperthreaded setups.

For more information on managing this feature, see Simultaneous Multithreading and TMM Values sections.

Summary of the CVE¶

CVE, short for Common Vulnerabilities and Exposures, is a list of publicly disclosed computer security flaws. When someone refers to a CVE, they mean a security flaw that’s been assigned a CVE ID number.

How to check the version of RH has been patched for a CVE¶

There are a few ways that you may check if a specific kernel has been patched for a specific CVE. Here are a few of them. If you have the rpm, you could use the rpm command to check the change log and grep for the CVE name.

Example:

rpm -qp kernel-3.10.0-862.11.6.el7.x86_64.rpm --changelog | grep CVE-2017-12190

If the kernel package for the kernel in question is in a repo that is configured and enabled on your server, you could use yum as follows:

yum list --cve CVE-2017-12190 | grep kernel.x86_64
kernel.x86_64                    3.10.0-327.22.2.el7     @rhel-7-server-eus-rpms
kernel.x86_64                    3.10.0-514.2.2.el7      @rhel-7-server-rpms    
kernel.x86_64                    3.10.0-693.2.2.el7      @rhel-7-server-rpms    
kernel.x86_64                    3.10.0-862.14.4.el7     rhel-7-server-rpms 

This shows that the above kernels include patches for CVE CVE-2017-12190

Verifying the version of OpenShift is vulnerable to the CVEs¶

Red Hat OpenShift Container Platform (RHOCP) 4 includes a fully managed node operating system, Red Hat Enterprise Linux CoreOS, commonly referred to as RHCOS.

The OpenShift cluster updates RHCOS when applying cluster updates, including sometimes updating between RHEL minor releases. This details the underlying RHEL minor versions present in currently supported versions of OCP.

OpenShift uses RHCOS (Red Hat CoreOs). This table shows which version of RHCOS maps to which RHEL version:

RHCOS/OCP Versions  RHEL Versions
6           RHEL  8.2
7.0-4.7.23  RHEL  8.3
7.24+       RHEL  8.4
8           RHEL  8.4
9           RHEL  8.4
10          RHEL  8.4
11          RHEL  8.6
12          RHEL  8.6
13          RHEL  9.2

OCP 4.12 uses RHEL 8.6. The above patches were applied on RHEL8, which implies the above patches are available in OCP 4.12.

Mitigations¶

Mitigations are generally to apply Intel microcode patches, Kernel/OS patches, or else compiler mitigations such as “return trampoline” (aka retpoline).
Another common mitigation is to make sure that no untrusted processes/Deployment is sharing the same CPU; configure Deployment to use 2 or more “whole cores”.
When full-pcpus-only policy option is specified along with static CPU Manager policy, an additional check in the allocation logic of the static policy ensures that CPUs would be allocated such that full cores are allocated. Because of this check, a pod would never have to acquire single threads with the aim to fill partially-allocated cores.
If the kubernetes instance configured properly with static CPU policy and full-pcpus-only policy, then when TMM starts with correct count of CPU resources, for example 2 it will be assigned both threads of the same core which means it will be assigned the whole core. No thread for the same core should be assigned to other work-load.
However, it is still dependent on how kubernetes is configured, and how TMM starts. There should be two stages here.
1. When all configured properly and the TMM has two threads of the same core. At the moment, mapres will detect these two threads as separate cores and will therefore start 2 tmm threads. This initial mode should have a warning for performance implications.
2. Future implementation, when TMM starts with two threads, mapres (or similar) detects SMT, validates threads belong to the same core, and only uses 1 of these threads. i.e. starts only a single TMM thread per core. Effectively, utilizing the entire core.

Persistent storage¶

The required Fluentd logging collector, dSSM database and Traffic Management Microkernel (TMM) Debug Sidecar require an available Kubernetes persistent storage to bind to during installation.

Next¶

Continue to the Getting Started guide to begin integrating the SPK software components.

Feedback¶

Provide feedback to improve this document by emailing spkdocs@f5.com.

Supplemental¶

The CNI project.
SPK Networking Overview.

Previous Next