Secure SPK Deployment On Secondary Interfaces Using Macvlan


Service Proxy for Kubernetes (SPK) can apply security functions using Firewall CRDs to ingress and egress traffic on your application. This only applies to traffic on the default interface, eth0, of your applications. But you might need to also apply security functions to an additional interface on your application. This guide walks you through enabling SPK firewall on the secondary network interface.

info Note: The SPK v1.9.0 Multus Capture is considered Early Access (EA). EA features are unsupported and are made available to get customer feedback on feature functionality and stability.

The firewall CRDs can be found in the SPK Firewall CRDs: Secure SPK section of the SPK CRs page


Ensure you have:

  • Installed the SPK Cert Manager
  • Installed the SPK Software
  • Installed the SPK CWC
  • Installed the RabbitMQ Pod
  • Installed Licensed SPK
  • Installed SPK Controller
  • Deployed your application in SPK watch namespace
    • TCP App used as example in this guide
    • A secondary interface using macvlan
  • Configured SPK network specific configurations, such as F5SPKVlan
  • Access to K8S Cluster with SPK

High Level Network Overview

When you add an additional network interface on a pod using the Multus CNI, traffic going to and from that interface bypasses the default CNI. And for us, it also bypasses SPK. An example illustrating this is shown below. Multus Traffic Overview Diagram 1: Traffic on Additional Pod Interfaces

In Diagram 1, traffic from a pod, by default, uses the default CNI for network traffic. In the example here, the default network traffic is through eth0. But when you add an additional interface, say net1, traffic will bypass the default CNI and thus SPK. And in the case here, it is connected to a router. Because of the different network route, you will not be able to apply any SPK security features on net1 by default.

This guide will walk through steps to change network traffic on the secondary interface so that it goes though SPK enabling firewall functions.

info This release will route traffic from only ONE additional interface on your application through SPK.

We will assume the following SPK configuration and the overall infrastructure. The network IP address examples throughout this guide will be also be based from the diagram below.

Firewall Infrastructure Overview Using Multus Diagram 2: SPK Firewall Infrastructure Overview Using Multus

info The External Client has a static route configured such that is routed to the external interface of SPK. A snippet showing this route below.

external-client $ ip route via dev ens224 proto static
. . .

Secure SPK for Applications Using Secondary Interface

Before You Begin

  1. You will need admin access to the worker nodes where your application pods are deployed.
  2. Confirm the CSRC container image, spk-csrc, is uploaded to your image registry. Refer to the Upload the image section of the SPK Software for more information.

Environment Setup

In order to enable the additional interface on your application so traffic is routed through SPK, use the instructions below to setup the networking requirements.

(OpenShift) Enable Route Via Host

OpenShift’s CNI, OVN-Kubernetes, needs to enable routingViaHost. Below is a snippet of that shows how to enable this feature.

  • Run oc edit to confirm it is enabled.

    kind: Network
            routingViaHost: true

Create Additional Macvlan Shim Interface

The shim interface is an additional Macvlan bridge interface created on the worker nodes where your applications are deployed. This is required so your secondary Macvlan interface have connectivity SPK’s internal interface. Take the following steps to create this:

  1. Establish SSH connection to the worker node where your application is deployed.

  2. Determine the network interface that corresponds to SPK’s internal network. In this case here, it is br-ex.

  3. Create the shim interface based from the interface used for SPK’s internal network.

    ip link add spk-shim link br-ex type macvlan mode bridge
  4. Assign an IP to the new spk-shim shim interface.

    ip addr add dev spk-shim
  5. Bring up the shim interface

    ip link set spk-shim up

Create Static Route Through Shim Interface

Traffic going to SPK from your application pod must be routed through the shim interface. This can be done using a static route. Below is an example how you can configure this using the NetworkAttachmentDefinition CR specified in ipam.routes.

  1. is the subnet configured on the external client.
  2. is the configured IP address on the shim interface.
apiVersion: ""
kind: NetworkAttachmentDefinition
  name: macvlan-secondary-interface
  config: '{
            "cniVersion": "0.3.1",
            "type": "macvlan",
            "mode": "bridge",
            "master": "ens224",
            "ipam": {
              "type": "host-local",
              "ranges": [
                [ {
                    "subnet": ""
                } ]
              "routes": [
                { "dst": "", "gw": "" }

Deploy Calico Static Route Configuration (CSRC) Daemonset to Enable Egress

The CSRC Daemonset needs to be installed in order to enable egress traffic on the secondary interface of your application. This daemonset enables it by creating kernel rules and routes. Be sure to have the following settings configured on install.

Setting Value Notes
daemonsetMode secureSPK
ipFamily ipv4 ipv4 or ipv6
f5MiniCniInstall no
spec.interfacename br-ex select the host interface for data traffic
spec.json.ipPoolCidrInfo.cidrList List List of cluster CIDR
spec.json.ipPoolCidrInfo.ipPoolList List List of IPPool CIDR
  1. Generate the values.yaml file to be used for CSRC install. Below is used in this example.

    • Each cluster-cidrN list entry corresponds to each worker node’s network CIDR. You can find this by running kubectl get node <nodeName> --output yaml and looking at
    • The app-ip-pool corresponds to the entire cluster’s pod network.
    daemonsetMode: secureSPK
    ipFamily: ipv4
      repository: <yourContainerImageRepo>
      interfacename: br-ex
            - name: cluster-cidr0
            - name: cluster-cidr1
            - name: app-ip-pool
    f5MiniCniInstall: no
  2. Install CSRC

    helm install mycsrc csrc-0.4.10.tgz -f values.yaml -n default
  3. You should see as many pods as you have worker nodes. In the example below, my cluster has 2 worker nodes so there are a total of two pods where each is deployed to a worker node.

    kubectl get pods -owide -n default
    NAME                READY   STATUS    RESTARTS   AGE    IP           NODE                      NOMINATED NODE   READINESS GATES
    f5-spk-csrc-4l6sc   1/1     Running   0          4d1h   ocp-worker2   <none>           <none>
    f5-spk-csrc-hdwp4   1/1     Running   0          4d1h   -ocp-worker1   <none>           <none>

Enable SPK Firewall Functions on Application’s Secondary Interface

SPK firewall functions can be enabled on the secondary interface of your applications. It is done by adding the following labels to your applications that will result traffic from your secondary interface being routed to SPK. This can be enabled on a per application basis. Take the following steps to enable firewall functions on your secondary interface.

  1. Enable Secure SPK

    kubectl label pods <applicationPod> enableSecureSPK="true"
  2. Provide the secondary interface name.

    kubectl label pods <applicationPod> secureSPKIfName=net1
  3. Provide the port to secure.

    kubectl label pods <applicationPod> secureSPKPort="8050"

You should now see the following labels when you describe your application.

kubectl describe pod <applicationPod>

. . .
Labels:           enableSecureSPK=true
                  . . .


Application Worker Nodes

You can confirm the egress rules and routes from CSRC are configured properly by checking them on the worker node.

  1. Establish SSH connection to your worker node where your application pod is running

  2. List the ip rules

    sh-4.4# ip rule list
    32276:  from lookup 276
    . . .
  3. List the route table that corresponds to the IP of the secondary interface

    sh-4.4# ip route show table 276
    default via dev br-ex via dev br-ex

Validating Routes on TMM

You can confirm if TMM have a route to the secondary interface in the f5-tmm pod.

  1. Go into the debug container from f5-tmm deployment

    oc exec -it f5-tmm-56944774c-49shg -c debug -- bash
  2. List routes

    /ip r via dev internal

Validating Virtual Servers on TMM

Once you’ve added the enableSecureSPK=true, secureSPKIfName, and secureSPKPort labels to the pod where you want to enable traffic of secondary interface so that it routes to SPK, you should see the following Virtual Servers created on the f5-tmm pod.

$ oc exec -it f5-tmm-56944774c-pdpql -c debug -- bash
/ tmctl -f /var/tmstat/blade/tmm0 virtual_server_stat -s name


Securing Traffic on Secondary Interface

You can now assign SPK Firewall functions to the secondary interface by defining firewall rules and policies. The global firewall rules you define will be applied to this traffic. Refer to the Secure SPK Deployment guide for additional details relating SPK Firewall.


Provide feedback to improve this document by emailing


  • The SPK Firewall CRDs: Secure SPK section of the SPK CRs guide.