BIG-IP YK VE on KVM in CN x86 CPU AND CN Linux Distribution

Overview

This document is a general overview of testing BIG-IP VE on a private infrastructure enabled by Kernel-based Virtual Machine (KVM) running in CN x86 CPUs and CN Linux Distribution. The testing was designed to provide the following results:

  • BIG-IP VE is compatible with running on Chinese x86 CPU. Specific models and details of the class of CPUS please refer to below details of CPU.
  • BIG-IP VE is compatible with running on open source openEuler Linux distribution. Specific version of the Linux variant pls refer to below details of Operating System.
  • BIG-IP VE system and its drivers are generally functional within generic Linux/KVM virtual machine.
  • SR-IOV capable Ethernet cards from the BIG-IP supported list are tested functional. Details of tested model please refer to below section.

The main BIG-IP version used in this document is vBIGIP-17.1.1, and other versions from the v17.x family are also expected to work.

Note: The purpose of this test is not to show any performance results.

Chinese x86 CPU

China x86 CPUs are a series of CPU products manufactured in China, codenamed Hygon. Chinese x86 CPUs are products of the Chinese company Haiguang, a joint venture between AMD and China. This CPU series is licensed and based on the Zen 1 core design. The reference on how the design is manufactured in China can be accessed on the AMD–Chinese_joint_venture page on Wikipedia.

openEuler Linux Distribution

openEuler is an open source operating system with free Linux distribution platform. Its repository and documentations is publicly accessible.

For lab OS info of the openEuler, use:

Command:

[root@localhost ~]# cat /etc/os-release

Response:

NAME="openEuler"

VERSION="22.03 (LTS-SP2)"

ID="openEuler"

VERSION_ID="22.03"

PRETTY_NAME="openEuler 22.03 (LTS-SP2)"

ANSI_COLOR="0;31

KVM virtualisation and hardware support in Linux Distribution

KVM is a software feature that you can install on physical Linux machines to create virtual machines. KVM acts as a virtual platform to run the BIG-IP YK VM.

Mod name Version
libvirt libvirt-6.2.0-57.oe2203sp2.x86_64
qemu qemu-6.2.0-74.oe2203sp2.x86_64
virt-install virt-install-2.1.0-8.oe2203sp2.noarch
virt-manager virt-manager-2.1.0-8.oe2203sp2.noarch
Test server specification example
Processor System CPU HYGON 3200 processor family
Chipset HYGON C86 3250 processor family
BIOS AMI UEFI BIOS
Memory Technology

Total 4 DIMMs (Eight channels, one DIMM per channel)

Support DDR4 2133~2666MHz ECC RDIMM/LRDIMM
Capacity 64GB (up to 256GB)
Expansion Onboard Expansion Slots Two PCIe Gen3 x8 slot for Aewin NIC module
Ethernet GbE

6 x RJ45 GbE, Intel I210,

4 x SFP Intel I350
10GbE 2 Slot 4 x SFP+, Intel x710
Storage SATA Support SATA Gen3 ；4x SATA
I/O USB

Two external USB 3.0 port

One internal USB 2.0 (5x2 pin header)
Serial One RJ45 Console port (COM1)
Power Supply Watt

1U: Redundant 300W AC-DC / DC-DC PSU (Option)

2U: 3Y 300W
Mechanical and Environmental Form Factor 1U 2/ 2U 2
LED One Power LED (Green), One HDD LED (Yellow), Two GPO (Green/Yellow)
Operating Temperature Operating: 0 ~ 40°C (32 ~ 104°F)
Storage Temperature -45 ~ 75°C (-49 ~ 167°F)
Humidity 5 ~ 95% relative humidity, non-operating, non-condensing

Hardware information list (hypervisor)

There are two types of hardware platforms integrated into the local product suite. From low-end to high-end, each has different CPUs and motherboard PCBs to achieve different target performance levels.

Hardware spec of low-end local server

Command:

[root@localhost ~]# lshw -short

Response:

H/W path    Device  Class   Description
/0/2f       processor   Hygon C86 3250 8-core Processor
/0/100/1.1/0    eth0    network  I210 Gigabit Network Connection
/0/100/1.2/0    GE1 network  I210 Gigabit Network Connection
/0/100/1.3/0    GE2 network  I210 Gigabit Network Connection
/0/100/1.4/0    GE3 network  I210 Gigabit Network Connection
/0/100/1.5/0    GE4 network  I210 Gigabit Network Connection
/0/100/1.6/0    GE5 network  I210 Gigabit Network Connection
/0/100/1.7/0    FE4 network  I350 Gigabit Fiber Network Connection
/0/100/1.7/0.1  FE3 network  I350 Gigabit Fiber Network Connection
/0/100/1.7/0.2  FE2 network  I350 Gigabit Fiber Network Connection
/0/100/1.7/0.3  FE1 network  I350 Gigabit Fiber Network Connection
/0/100/3.1/0    SE4 network  Ethernet Controller X710 for 10GbE SFP+
/0/100/3.1/0.1  SE3 network  Ethernet Controller X710 for 10GbE SFP+
/0/100/3.1/0.2  SE2 network  Ethernet Controller X710 for 10GbE SFP+
/0/100/3.1/0.3  SE1 network  Ethernet Controller X710 for 10GbE SFP+
/0/100/3.1/e    ens3f3v0    network  Ethernet Virtual Function 700 Series
/0/100/3.1/e.1  ens3f3v1    network  Ethernet Virtual Function 700 Series

Command:

[root@localhost ~]# lscpu

Response:

架构：                    x86_64
  CPU 运行模式：          32-bit, 64-bit
  Address sizes:          48 bits physical, 48 bits virtual
  字节序：                  Little Endian
CPU:    16
  在线 CPU 列表：            0-15
厂商 ID：                    HygonGenuine
  BIOS Vendor ID:            Chengdu Hygon
  型号名称：                 Hygon C86 3250  8-core Processor
    BIOS Model name:       Hygon C86 3250  8-core Processor
Virtualization features:
  虚拟化：                   AMD-V

Hardware specification of high-end local server

Command:

[root@localhost ~]# lshw -short

Response:

H/W path    Device      Class          Description
/0/8                                 processor        Hygon C86 7380 32-core Processor
/0/100/1.3/0            SE1          network          Ethernet Controller E810-C for QSFP
/0/100/1.3/0.1          SE2          network          Ethernet Controller E810-C for QSFP
/0/100/1.3/1                         network          Ethernet Adaptive Virtual Function
/0/100/1.3/11                        network          Ethernet Adaptive Virtual Function
/0/12e/1.6/0            eth1         network          I210 Gigabit Network Connection
/0/12e/1.7/0            eth2         network          I210 Gigabit Network Connection
/0/135/3.1/0            SE3          network          Ethernet Controller E810-C for QSFP
/0/135/3.1/0.1          SE4          network          Ethernet Controller E810-C for QSFP
/0/135/3.1/1                         network          Ethernet Adaptive Virtual Function
/0/135/3.1/11                        network          Ethernet Adaptive Virtual Function

Command:

[root@localhost ~]# lscpu

Response:

CPU 运行模式：          32-bit, 64-bit
  Address sizes:          48 bits physical, 48 bits virtual
  字节序：                Little Endian
CPU:    32
  在线 CPU 列表：            0-31
厂商 ID：                  HygonGenuine
  BIOS Vendor ID:            Chengdu Hygon
  型号名称：              Hygon C86 7380 32-core Processor
    BIOS Model name:       Hygon C86 7380 32-core Processor
Virtualization features:
  虚拟化：                   AMD-V

Lab OS information

To check OS Kernel:

Command:

[root@localhost ~]# uname -a

Response:

Linux localhost.localdomain 5.10.0-153.12.0.92.oe2203sp2.x86_64 #1 SMP Wed Jun 28 23:04:48 CST 2023 x86_64 x86_64 x86_64 GNU/Linux

To check OS release:

Command:

[root@localhost ~]# cat /etc/os-release

Response:

NAME="openEuler"
VERSION="22.03 (LTS-SP2)"
ID="openEuler"
VERSION_ID="22.03"
PRETTY_NAME="openEuler 22.03 (LTS-SP2)"
ANSI_COLOR="0;31"

Lab VM management console

Once logged into the openEuler hypervisor command line where virt-manager is installed, run virt-manager to acquire the gui config console.

Command:

[root@localhost ~]# uname -a

The virt-manager gui console helps to provide a visualized view of all components and parameters of the BIG-IP VE. including:

  • Number of cores of the vCPU allocated to VE and CPU archtitecture.
  • Number of GB memory allocated to VE.
  • Size of virtual Hard Disk(s).
  • Number of virtual Ethernet cards.
  • Number of SR-IOV Ethernet card VFs.
  • Serial port and other IO devices.
../_images/Lab_VM_management_console.png

Virtualisation software setup

Run the following command for Virtualisation Software components and versions:

Command

[root@localhost ~]# rpm -qa qemu*

Response:

qemu-hw-usb-host-6.2.0-74.oe2203sp2.x86_64
qemu-img-6.2.0-74.oe2203sp2.x86_64
qemu-block-iscsi-6.2.0-74.oe2203sp2.x86_64
qemu-block-curl-6.2.0-74.oe2203sp2.x86_64
qemu-6.2.0-74.oe2203sp2.x86_64
qemu-help-6.2.0-74.oe2203sp2.noarch

Command:

[root@localhost ~]# rpm -qa libvirt*

Response:

libvirt-bash-completion-6.2.0-57.oe2203sp2.x86_64
libvirt-libs-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-network-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-interface-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-nodedev-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-secret-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-qemu-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-config-network-6.2.0-57.oe2203sp2.x86_64
libvirt-client-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-nwfilter-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-config-nwfilter-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-core-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-disk-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-gluster-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-iscsi-direct-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-mpath-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-rbd-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-scsi-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-iscsi-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-logical-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-driver-storage-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-kvm-6.2.0-57.oe2203sp2.x86_64
libvirt-daemon-qemu-6.2.0-57.oe2203sp2.x86_64
libvirt-6.2.0-57.oe2203sp2.x86_64
libvirt-nss-6.2.0-57.oe2203sp2.x86_64
libvirt-lock-sanlock-6.2.0-57.oe2203sp2.x86_64
libvirt-admin-6.2.0-57.oe2203sp2.x86_64
libvirt-devel-6.2.0-57.oe2203sp2.x86_64
libvirt-docs-6.2.0-57.oe2203sp2.x86_64
libvirt-glib-4.0.0-2.oe2203sp2.x86_64

Command:

[root@localhost ~]# rpm -qa virt*

Response:

virt-what-1.21-1.oe2203sp2.x86_64
virt-manager-common-2.1.0-8.oe2203sp2.noarch
virt-install-2.1.0-8.oe2203sp2.noarch
virt-manager-2.1.0-8.oe2203sp2.noarch

SR-IOV Ethernet card details:

SR-IOV Ethernet card setup requires both PF (physical function) and VF (virtual function) drivers.

PF is implemented and loaded with default driver from the operating system in this lab. VF is an embedded driver from BIG-IP VE.

The target Single Root I/O Virtualisation (SR-IOV) Network Interface Card (NIC) for this test scope are from the inherent BIG-IP recommended list.

  • XL710 (40G)
  • E810 (100G)

For detailed information, refer BIG-IP VE NIC adapters that support SR-IOV.

Command:

[root@localhost ~]# lshw -c network -businfo

Response:

Bus info          Device     Class          Description
=======================================================
pci@0000:01:00.0  eth0       network        I210 Gigabit Network Connection
pci@0000:02:00.0  GE1        network        I210 Gigabit Network Connection
pci@0000:03:00.0  GE2        network        I210 Gigabit Network Connection
pci@0000:04:00.0  GE3        network        I210 Gigabit Network Connection
pci@0000:05:00.0  GE4        network        I210 Gigabit Network Connection
pci@0000:06:00.0  GE5        network        I210 Gigabit Network Connection
pci@0000:07:00.0  FE4        network        I350 Gigabit Fiber Network Connection
pci@0000:07:00.1  FE3        network        I350 Gigabit Fiber Network Connection
pci@0000:07:00.2  FE2        network        I350 Gigabit Fiber Network Connection
pci@0000:07:00.3  FE1        network        I350 Gigabit Fiber Network Connection
pci@0000:09:00.0  SE4        network        Ethernet Controller X710 for 10GbE SFP+
pci@0000:09:00.1  SE3        network        Ethernet Controller X710 for 10GbE SFP+
pci@0000:09:00.2  SE2        network        Ethernet Controller X710 for 10GbE SFP+
pci@0000:09:00.3  SE1        network        Ethernet Controller X710 for 10GbE SFP+
pci@0000:09:02.0             network        Ethernet Virtual Function 700 Series
pci@0000:09:06.0             network        Ethernet Virtual Function 700 Series
pci@0000:09:0a.0             network        Ethernet Virtual Function 700 Series

For details of how SR-IOV VF and its affiliated drivers are setup within BIG-IP VE, please refer to standard guide.

Once done SR-IOV VFs attached to the BIG-IP VE, can be viewed from virt-manager GUI console.

../_images/SR-IOV_VF.png

BIG-IP VE will be able to set the drivers for the specific VF. Here is what was verified in this lab, a ixlv driver. Use the “tmctl -d blade tmm/device_probed ” or “tmctl -d blade tmm/xnet/device_probed ” commands to list up. the drivers in use. Detail reference can be found on SR-IOV article.

../_images/KVM.png