Configure the IPsec pod network on a vanilla cluster

This document explains how to configure the Internet Protocol Security (IPsec) pod’s network interface on a vanilla (non-OpenShift) Kubernetes cluster that uses Intel NICs with vfio-pci device type for the Traffic Management Microkernel (TMM) pods.

The IPsec pod requires a kernel-visible netdevice interface. On vanilla clusters where TMM uses vfio-pci Virtual Functions (VFs), you must dedicate a small subset of VFs as netdevice type specifically for the IPsec pod. The IPsec pod’s IP address must be on the same subnet as TMM’s internal self-IP so the two pods can exchange IKE and ESP traffic.

Before you begin

Make sure you have:

• A Kubernetes cluster with CNF 2.4 or later installed

• SR-IOV Network Operator deployed and configured

• TMM pods running with vfio-pci SR-IOV interfaces

• kubectl access to the cluster with permissions to create SR-IOV network resources and Helm releases

• The subnet and IP address range used by TMM’s internal self-IP (for example, 192.168.20.208/28)

  Note

  The IPsec pod’s IP address and CIDR prefix length must match the subnet used by TMM’s self-IP on the shared interface.

Create a netdevice SR-IOV pool for IPsec

Reserve a subset of VFs on the NIC as netdevice type. This creates a separate resource pool from TMM’s vfio-pci pool and gives the IPsec pod a kernel-visible interface.

In the following example, VFs 28–31 on the ens2f0 physical function are reserved as netdevice:

1. Create a YAML file named sriov-ipsec-netdevice.yaml:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: sriov-nnp-ens2f0-netdevice
     namespace: sriov-network-operator
   spec:
     deviceType: netdevice
     nicSelector:
       deviceID: "1592"
       pfNames:
         - ens2f0#28-31
       vendor: "8086"
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     numVfs: 32
     resourceName: ens2f0-netdevice
   

Adjust the pfNames, deviceID, vendor, and VF range to match your NIC hardware. The VF range (for example, 28-31) must not overlap with VFs assigned to TMM’s vfio-pci pool.

2. Apply the SR-IOV network node policy:

   kubectl apply -f sriov-ipsec-netdevice.yaml
   

3. Verify that the SR-IOV network node policy was applied successfully:

   kubectl get sriovnetworknodepolicy -n sriov-network-operator
   

Create a NetworkAttachmentDefinition

Define a Multus network that pulls VFs from the netdevice resource pool you created. The IP subnet must match TMM’s internal self-IP subnet so both pods can route to each other.

1. Create a YAML file named nad-ipsec-netdevice.yaml:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: internal-netdevice
  namespace:
  annotations:
    k8s.v1.cni.cncf.io/resourceName: openshift.io/ens2f0-netdevice
spec:
  config: |-
    {
      "cniVersion": "1.0.0",
      "name": "internal-netdevice",
      "type": "sriov",
      "ipam": {
        "type": "host-local",
        "ranges": [
          [
            {
              "subnet": "10.10.10.0/24",
              "rangeStart": "10.10.10.1",
              "rangeEnd": "10.10.10.254"
            }
          ],
          [
            {
              "subnet": "2001:db8::/112",
              "rangeStart": "2001:db8::1",
              "rangeEnd": "2001:db8::fffe"
            }
          ]
        ]
      }
    }

Replace the subnet, rangeStart, and rangeEnd values with the IP range that matches your TMM internal self-IP subnet.

2. Apply the NetworkAttachmentDefinition:

   kubectl apply -f nad-ipsec-netdevice.yaml
   

3. Verify the resource was created:

   kubectl get network-attachment-definition -n f5ingress
   

Update the IPsec Helm values

• Configure the IPsec pod to attach to the internal-netdevice network, assign it a static IP address on TMM’s internal subnet, and request a VF from the netdevice resource pool. Add the following to your overrides file:

  f5-ipsec:
    enabled: true
  
    ipsec:
      name: f5-ipsec
      enabled: true
  
      image:
        repository: repo.f5.com/images
  
      # SR-IOV network attachment — name must match SriovNetwork resource deployed in your cluster
      annotations:
        k8s.v1.cni.cncf.io/networks: |
          [
            {
              "name": "internal-netdevice",
              "ips": ["10.10.10.1/24", "2001:db8::1/112"]
            }
          ]
  
      # DSSM namespace must match where dssm-f5-dssm is deployed — wrong namespace breaks HA store on restarts
      dssm:
        service: dssm-f5-dssm-sentinel.<namespace>   # replace <namespace> with actual DSSM namespace
  
      fluentbit_sidecar:
        image:
          repository: repo.f5.com/images
        fluentd: 
          host: f5-toda-fluentd.<namespace>.svc.cluster.local.   # replace <namespace> with actual fluentd namespace
      
      resources:
        requests:
          sriovResources:
            openshift.io/ens2f0-netdevice: "1"
        limits:
          sriovResources:
            openshift.io/ens2f0-netdevice: "1"
  

Confirm subnet alignment

After the IPsec pod starts, verify that its IP address is on the same subnet as TMM’s internal self-IP:

Component	Expected IP (example)	Subnet
TMM internal self-IP	10.10.10.2	10.10.10.2/24
IPsec pod IP	10.10.10.1	10.10.10.1/24

Both addresses must be on the same subnet so the two pods can route IKE and ESP traffic to each other.