AdminPolicyBasedExternalRoute CR¶

Introduction to Multiple External Gateway

The Multiple External Gateway (MEG) is designed to enhance how clusters connect to external networks by enabling the use of multiple gateways to manage traffic between the cluster and the external entities.

The AdminPolicyBasedExternalRoute Custom Resource (CR) allows admin users to configure OpenShift to utilize the MEG feature. This CR controls how egress traffic from specific namespaces in OpenShift is routed through the Traffic Management Microkernel (TMM) and is critical for managing external traffic across multiple gateways when using SPK. For the full list of CRs, refer to the SPK CRs overview.

AdminPolicyBasedExternalRoute CR Parameters¶

The table below describes the AdminPolicyBasedExternalRoute CR parameters used in this document.

spec¶

The table below describes the CR spec parameters.

Parameter	Description
`from`	Specifies the source of the egress traffic (i.e., where the traffic originates).
`from.namespaceSelector.matchLabels`	Select namespaces based on their labels. For example, it filters namespaces with the label `kubernetes.io/metadata.name:spk-apps`, ensuring that the policy only applies to traffic originating from the specified namespace.
`nextHops`	Defines the destinations or intermediate points where the selected traffic will be forwarded. It can use dynamic pod selectors based on labels.
`nextHops.dynamic`	Specifies dynamic next hops based on pod and namespace selectors.
`nextHops.dynamic.podSelector.matchLabels`	Select pods labeled with `app: f5-tmm`. This indicates that traffic will be forwarded to pods belonging to the F5 Traffic Management Microkernel (TMM).
`nextHops.dynamic.namespaceSelector.matchLabels`	Select namespaces labeled with `kubernetes.io/metadata.name: spk-ingress`, ensuring that the policy routes traffic to pods in the specific namespace `spk-ingress`.
`nextHops.dynamic.networkAttachmentName`	Defines the specific network attachment to use for routing the traffic. For example,vlan-internal refers to the name of the network attachment that connects the TMM to the cluster's internal VLAN.

AdminPolicyBasedExternalRoute CR Example¶

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: spk-egress-policy
spec:
  from:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: spk-apps
  nextHops:
    dynamic:
      - podSelector:
          matchLabels:
            app: f5-tmm
        namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: spk-ingress
        networkAttachmentName: vlan-internal

To extend this configuration to various namespaces, replicate this configuration with modified values. As an example, to apply it to the spk-apps2 namespace:

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: spk-egress-policy2
spec:
  from:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: spk-apps2
  nextHops:
    dynamic:
      - podSelector:
          matchLabels:
            app: f5-tmm
        namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: spk-ingress
        networkAttachmentName: vlan-internal

Requirements¶

Ensure you have:

OCP versions 4.16+.
A Linux-based workstation.

Configuring AdminPolicyBasedExternalRoute CR¶

Follow the instructions to configure and apply the AdminPolicyBasedExternalRoute CR to route egress traffic in a Kubernetes/OpenShift cluster:

1. Modify the SPK overrides YAML file to set:

A. The tmm.ovn_meg.enabled parameter to true to enable multiple external gateway support.

f5-tmm:
  tmm: 
    ovn_meg:
      enabled: true   

B. (Optional) The tmm.icni2.enabled parameter to false to fully utilize AdminPolicyBasedExternalRoute. If ICNI 2.0 is supported by your OCP version, set to true for redundancy.

f5-tmm:
  tmm: 
    icni2:
      enabled: false

Important Note: When both the tmm.ovn_meg.enabled and tmm.icni2.enabled parameters are set to true, the AdminPolicyBasedExternalRoute CR of Multiple External Gateway takes precedence over ICNI 2.0.

2. Copy the below example AdminPolicyBasedExternalRoute CR to a YAML file and save.

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: spk-egress-policy
spec:
  from:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: spk-apps
  nextHops:
    dynamic:
      - podSelector:
          matchLabels:
            app: f5-tmm
        namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: spk-ingress
        networkAttachmentName: vlan-internal

3. Modify the values in the example CR to match your configuration requirements and save.

spec.from.namespaceSelector.matchLabels:
- Replace spk-apps with the name of your namespace(s) where the traffic originates (e.g., spk-apps , spk-apps2).
spec.nextHops.dynamic:
- Specify the namespace where your f5-tmm pods reside. ( e.g., spk-ingress ).A Linux-based workstation.
Ensure the podSelector.matchLabels matches app:f5-tmm.
networkAttachmentName:
- Replace vlan-internal with your internal VLAN name as specified in the F5SPKVlan CR.

4. Apply the AdminPolicyBasedExternalRoute CR that you have created.

oc apply -f externalroute.yaml 

Sample Output

adminpolicybasedexternalroute.k8s.f5net.com/externalroute created

Note: The AdminPolicyBasedExternalRoute CR applies globally to the cluster and affects egress traffic across all namespaces not just a specific namespace.

5. Verify if your configuration is applied successfully.

oc describe AdminPolicyBasedExternalRoute spk-egress-policy

The configuration status should display Success. You should see a message similar to:

Configured external gateway IPs: <internal VLAN IPs of F5-TMM pods>

If the status is Fail, check the following:

Verify errors in your configuration (e.g., namespace, pod labels, network attachment).
Ensure the following components are properly configured:
- Internal VLAN (F5SPKVlan CR).
- Egress Type CR (F5SPKEgress CR).

After fixing errors and adding missing configurations, reapply the AdminPolicyBasedExternalRoute CR.

Feedback

Provide feedback to improve this document by emailing spkdocs@f5.com.