Network configuration and application traffic management

Dynamic routing with ZebOS

BNK supports dynamic routing through ZebOS. You configure ZebOS routing features such as BGP, OSPF, and static routes by creating a ConfigMap. TMM loads the routing configuration from this file.

1. Create a ZebOS ConfigMap.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: zebos-config
     namespace: <cneInstanceNamespace>
   data:
     zebos.conf: |
       router bgp 65001
       neighbor 10.19.1.1 remote-as 65000
       address-family ipv4 unicast
       redistribute connected
       exit-address-family
   

   kubectl apply -f zebos-config.yaml --namespace <cneInstanceNamespace>
   

2. Verify the configuration.

   kubectl get configmap zebos-config --namespace <cneInstanceNamespace> -o yaml
   

Gateway API setup

BNK supports Gateway API v1.5.0. The CNE Controller manages Gateway API custom resources and F5 adapter resources, including SecPolicy and NetPolicy. These resources extend traffic, security, and network behavior.

CR	Function
GatewayClass	Defines the Gateway class that BNK implements.
Gateway	Acts as the cluster entry point for external traffic. Manages listeners.
HTTPRoute	Routes HTTP traffic by path, header, or weight.
GRPCRoute	Routes gRPC traffic.
ReferenceGrant	Enables cross-namespace references.
SecPolicy	Attaches firewall or DDoS policies to a Gateway or GatewayClass.
NetPolicy	Attaches iRules, TCP settings, and HSL settings to a Gateway.
BackendTLSPolicy	Configures TLS from the Gateway to backend Services. See Secure connections to backends.

Prerequisites

Client-side and server-side mutual TLS (mTLS) require Gateway API standard bundle version 1.5 or later.

The following table lists the minimum Gateway API bundle version for each capability:

Capability	Minimum bundle version
Client-side TLS	1.0
Client-side mTLS	1.5
Server-side TLS	1.4
Server-side mTLS	1.5

Upgrade the Gateway API bundle on IBM ROKS

IBM Red Hat OpenShift on IBM Cloud (ROKS) clusters include a pre-installed Gateway API bundle. If the installed version is older than 1.5, run the following commands to upgrade to version 1.5.0.

Note

You might need to repeat these steps if the validation webhook blocks the operation.

oc delete validatingadmissionpolicybinding openshift-ingress-operator-gatewayapi-crd-admission
oc delete validatingadmissionpolicy openshift-ingress-operator-gatewayapi-crd-admission
kubectl apply --server-side --force-conflicts -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yaml

Server-side TLS

A Gateway sits between two separate TLS connections. Each TLS connection is configured independently.

Connection	Gateway’s role	Configured with
External client → Gateway	Server	listeners[].tls in the Gateway resource. See Ingress traffic with HTTPRoute and GRPCRoute.
Gateway → backend Service	Client	BackendTLSPolicy and tls.backend in the Gateway resource, described in this section.

This section covers the connection between the gateway (TMM) and the server. Terminating TLS for incoming client traffic does not encrypt traffic onward to your backends — you configure that separately here.

Two resources are involved:

To do this	Use
Verify the backend’s certificate	BackendTLSPolicy
Present the Gateway’s certificate to the backend	tls.backend.clientCertificateRef in the Gateway resource

Each resource on its own gives you one-directional TLS. Use both together for mutual TLS (mTLS): the Gateway proves its identity to the server, and the server proves its identity to the Gateway.

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: BackendTLSPolicy
metadata:
  name: backend-tls-example
  namespace: default
spec:
  targetRefs:
  - group: ""
    kind: Service
    name: my-backend-service
    port: 8443
  validation:
    caCertificateRefs:
    - name: ca-cert-secret
      group: ""
      kind: ConfigMap
    hostname: backend.example.com
status:
  ancestors:
  - ancestorRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: example-gateway
      namespace: default
      port: 443
    controllerName: f5.com/bnk-ns-cne-controller
    conditions:
    - lastTransitionTime: "2026-06-09T10:30:00Z"
      message: "Policy accepted by controller"
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2026-06-09T10:30:05Z"
      message: "Policy programmed to gateway"
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
  conditions:
  - lastTransitionTime: "2026-06-09T10:30:00Z"
    message: "BackendTLSPolicy has been accepted"
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2026-06-09T10:30:05Z"
    message: "CA certificate resolved successfully"
    observedGeneration: 1
    reason: ResolvedRefs
    status: "True"
    type: ResolvedRefs
  observedGeneration: 1

Note

The Certificate Authority (CA) bundle cannot be sent directly with SSL profile objects via the TMM async API because of its size; it must be stored in the TMM container. Use an inline CA bundle to achieve this. The bundle is transferred inline within the async API object, which removes the need for a separate file transfer mechanism.

Client-side TLS

Connections between the gateway (TMM) and the client are configured using listeners at the Gateway level.

Listeners and TLS

Listeners expose the TLS setting on a per-domain or subdomain basis. TLS settings of a listener are applied to all domains that satisfy the hostname criteria.

In the following example, the Gateway serves the TLS certificate defined in the default-cert Secret resource for all requests.

listeners:
- protocol: HTTPS # Other possible value is `TLS`
  port: 443
  tls:
    mode: Terminate # If protocol is `TLS`, `Passthrough` is another possible mode
    certificateRefs:
    - kind: Secret
      group: ""
      name: default-cert

Listeners with different certificates

In this example, the Gateway is configured to serve the foo.example.com and bar.example.com domains. The certificate for these domains is specified in the Gateway.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: tls-basic
spec:
  gatewayClassName: example
  listeners:
  - name: foo-https
    protocol: HTTPS
    port: 443
    hostname: foo.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: foo-example-com-cert
  - name: bar-https
    protocol: HTTPS
    port: 443
    hostname: bar.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: bar-example-com-cert

Wildcard TLS listeners

In this example, the Gateway is configured with a wildcard certificate for *.example.com and a different certificate for foo.example.com. Since a specific match takes priority, the Gateway will serve foo-example-com-cert for requests to foo.example.com and wildcard-example-com-cert for all other requests.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: wildcard-tls-gateway
spec:
  gatewayClassName: example
  listeners:
  - name: foo-https
    protocol: HTTPS
    port: 443
    hostname: foo.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: foo-example-com-cert
  - name: wildcard-https
    protocol: HTTPS
    port: 443
    hostname: "*.example.com"
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: wildcard-example-com-cert

Caution

Don’t create a listener without a hostname for client-side TLS. A listener without a hostname accepts all Server Name Indication (SNI) values, but the certificates might not match. This can cause the wrong certificate to be served to clients, which creates a security risk.

Trust a specific CA

apiVersion: gateway.networking.k8s.io/v1
kind: BackendTLSPolicy
metadata:
  name: backend-tls-ca
  namespace: f5-alpha
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: backend-service
      sectionName: https              # optional: target a specific port by name
  validation:
    caCertificateRefs:
      - group: ""
        kind: ConfigMap
        name: ca-configmap            # must contain key 'ca.crt'
    hostname: my-backend.example.com   # SNI hostname used on the backend connection
    subjectAltNames:                   # optional: enforce SAN validation
      - type: Hostname
        hostname: backend-tls.example.com
      - type: URI
        uri: <uri>
  options: {}                          # implementation-specific TLS options

Trust the well-known CA set

  apiVersion: gateway.networking.k8s.io/v1
  kind: BackendTLSPolicy
  metadata:
    name: backend-tls-system
    namespace: f5-alpha
  spec:
    targetRefs:
      - group: ""
        kind: Service
        name: my-backend-service
    validation:
      wellKnownCACertificates: System
      hostname: my-backend.example.com

Set a minimum TLS version and cipher suites

spec.options accepts implementation-specific settings for the backend connection.

apiVersion: gateway.networking.k8s.io/v1
kind: BackendTLSPolicy
metadata:
  name: backend-tls-secure-ciphers
  namespace: default
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: api-backend
  validation:
    hostname: backend.example.com
    wellKnownCACertificates: System
  options:
    f5.com/ciphers: "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:!aNULL:!eNULL"
    f5.com/minTLSVersion: "1.2"

BackendTLSPolicy parameters

Parameter	Description	Implementation details
spec.options	Additional TLS settings for the backend connection. Use f5.com/ciphers for a cipher list in OpenSSL format, and f5.com/minTLSVersion for the minimum TLS version (1.2 or 1.3).	As per the Gateway API community standard.

CR status codes

The controller writes the following conditions back to .status.ancestors[].conditions:

Condition	Reason	Meaning
Accepted	Accepted	Policy accepted and programmed.
Accepted	Invalid	Policy spec is invalid (e.g., unsupported WellKnownCACertificates value).
Accepted	NoValidCACertificate	All CACertificateRefs are invalid.
Accepted	Conflicted	Another BackendTLSPolicy takes precedence for this target.
ResolvedRefs	ResolvedRefs	All references resolved successfully.
ResolvedRefs	InvalidCACertificateRef	A CACertificateRef could not be resolved.
ResolvedRefs	InvalidKind	A referenced resource kind is unknown or unsupported.

CR Parameters

The table below provides the description and implementation details of Gateway CR spec parameters.

Parameters	Description	Implementation details
spec.options	Provides additional TLS settings. Specify f5.com/ciphers to provide cipher list format defined by OpenSSL, and f5.com/minTLSVersion to specify the minimum TLS version allowed (1.2 or 1.3).	As per the Gateway API community standard.

Sample CRs

This section lists the sample CRs of use cases available to apply Gateway CR. In each sample, necessary parameters for applying a CR is added.

• Gateway CR with minimum TLS version and cipher suites

Gateway CR with minimum TLS version and cipher suites

Gateway TLS configurations provide an options map to add additional TLS settings for server-side implementation-specific features.

apiVersion: gateway.networking.k8s.io/v1
kind: BackendTLSPolicy
metadata:
  name: backend-tls-secure-ciphers
  namespace: default
spec:
  targetRefs:
    - group: ""
      kind: Service
      name: api-backend
  validation:
    hostname: backend.example.com
    wellKnownCACertificates: system
  # Implementation-specific TLS options with OpenSSL cipher format
  options:
    # OpenSSL cipher list for backend connection
    f5.com/ciphers: "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:!aNULL:!eNULL"
    f5.com/minTLSVersion: "1.2"

Client-side mTLS

Gateway API supports validating the Transport Layer Security (TLS) certificate that a frontend client presents during the TLS handshake.

Why validation is configured at the Gateway level

Server certificate configuration is defined per listener. Client certificate validation is different—it’s configured at the Gateway level in the spec.tls field.

This design prevents a security risk caused by HTTP/2 and TLS connection coalescing. Without Gateway-level validation, a connection established for one listener could be reused by another listener on the same port. This could bypass listener-specific validation settings.

Note

This feature is part of extended support. For more information on support levels, refer to the conformance guide.

Frontend validation settings

Client validation uses the frontendValidation struct to define how the Gateway verifies a client’s identity. The struct includes two fields:

• caCertificateRefs: A list of references to Kubernetes objects (typically ConfigMaps) that contain Privacy-Enhanced Mail (PEM)-encoded Certificate Authority (CA) certificate bundles. The Gateway uses these bundles as trust anchors to validate the client’s certificate.

• mode: Defines the validation behavior:

  • AllowValidOnly (default): The Gateway accepts connections only when the client presents a valid certificate that passes validation against the CA bundle.

  • AllowInsecureFallback: The Gateway accepts connections even when the client certificate is missing or fails verification. This mode delegates authorization to the backend.

Caution

Use AllowInsecureFallback with care. Connections that skip certificate validation rely entirely on the backend for authorization.

Gateway-level scoping

You can apply validation globally to the Gateway or override it for specific ports:

1. Default configuration: Applies to all HTTPS listeners on the Gateway unless a per-port override exists.

2. Per-port configuration: Overrides the default configuration for all listeners that handle traffic on a specific port. Use this option for fine-grained control.

The following example shows how to configure client certificate validation with a default configuration and a per-port override.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: client-validation-basic
spec:
  gatewayClassName: acme-lb
  tls:
    frontend:
      default:
        validation:
          caCertificateRefs
        
    :
          - kind: ConfigMap
            group: ""
            name: foo-example-com-ca-cert
      perPort:
      - port: 8443
        tls:
          validation:
            caCertificateRefs:
            - kind: ConfigMap
              group: ""
              name: bar-example-com-ca-cert
  listeners:
  - name: foo-https
    protocol: HTTPS
    port: 443
    hostname: foo.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: foo-example-com-cert
  - name: bar-https
    protocol: HTTPS
    port: 8443
    hostname: bar.example.com
    tls:
      certificateRefs:
      - kind: Secret
        group: ""
        name: bar-example-com-cert

Ingress traffic with HTTPRoute and GRPCRoute

1. Apply a Gateway CR.

   Create the entry point for external traffic.

   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: my-gateway
     namespace: <applicationNamespace>
   spec:
     gatewayClassName: f5-bnk-gateway-class
     listeners:
       - name: http
         port: 80
         protocol: HTTP
       - name: https
         port: 443
         protocol: HTTPS
         tls:
           mode: Terminate
           certificateRefs:
             - name: my-tls-secret
   

   kubectl apply -f gateway.yaml
   

2. Create an HTTPRoute.

   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: app-route
     namespace: <applicationNamespace>
   spec:
     parentRefs:
       - name: my-gateway
     rules:
       - matches:
           - path:
               type: PathPrefix
               value: /api
         backendRefs:
           - name: api-service
             port: 8080
   

   kubectl apply -f httproute.yaml
   

iRules integration (NetPolicy)

apiVersion: k8s.f5.com/v1
kind: L4Route
metadata:
  name: tcp-app
  namespace: <applicationNamespace>
spec:
  host: 11.19.1.100
  port: 8050
  protocol: TCP
  pool:
    service: backend-svc
    servicePort: 8050

kubectl apply -f l4route.yaml

Egress traffic with SNAT

1. Create a SNAT pool.

   apiVersion: k8s.f5net.com/v1
   kind: F5SPKSnatpool
   metadata:
     name: egress-snat
     namespace: <applicationNamespace>
   spec:
     addresses:
       - 11.19.1.200
       - 11.19.1.201
   

   kubectl apply -f snatpool.yaml
   

2. Create an Egress CR.

   apiVersion: k8s.f5net.com/v1
   kind: F5SPKEgress
   metadata:
     name: app-egress
     namespace: <applicationNamespace>
   spec:
     snatpoolName: egress-snat
   

   kubectl apply -f egress.yaml
   

iRules integration (NetPolicy)

Attach iRules to Gateway listeners with the NetPolicy resource. This allows custom traffic modification at the Gateway layer.

1. Create an iRule CR.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigCneIrule
   metadata:
     name: my-irule
     namespace: <applicationNamespace>
   spec:
     irule: |
       when HTTP_REQUEST {
         HTTP::header insert X-Forwarded-By BNK
       }
   

   kubectl apply -f irule.yaml
   

2. Create a NetPolicy that references the iRule.

   apiVersion: k8s.f5.com/v1
   kind: NetPolicy
   metadata:
     name: irule-policy
     namespace: <applicationNamespace>
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: Gateway
       name: my-gateway
     irules:
       - name: my-irule
   

   kubectl apply -f netpolicy.yaml
   

F5 IPAM Controller

The F5 IPAM Controller (FIC) manages IP address allocation for Gateway CRs.

Use case	When	Behavior
IP defined in Gateway	Gateway sets the address	FIC validates the IP and assigns it from the matching pool.
IP not defined	Gateway omits the address	FIC auto-assigns an available IP from the pool.

Firewall Policy (F5BigFwPolicy)

BNK provides a stateful firewall through the F5BigFwPolicy custom resource. You attach firewall policies to Gateway resources with SecPolicy.

1. Create an address list.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigCneAddresslist
   metadata:
     name: allowed-ips
     namespace: <applicationNamespace>
   spec:
     addresses:
       - 10.0.0.0/8
       - 192.168.0.0/16
   

   kubectl apply -f addresslist.yaml
   

2. Create a firewall rule list.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigFwRulelist
   metadata:
     name: allow-internal
     namespace: <applicationNamespace>
   spec:
     rules:
       - name: allow-internal-traffic
         action: accept
         source:
           addressLists:
             - allowed-ips
   

   kubectl apply -f fw-rulelist.yaml
   

3. Create a firewall policy.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigFwPolicy
   metadata:
     name: my-fw-policy
     namespace: <applicationNamespace>
   spec:
     rules:
       - rulelist: allow-internal
   

   kubectl apply -f fw-policy.yaml
   

4. Attach the policy to a Gateway with SecPolicy.

   apiVersion: k8s.f5.com/v1
   kind: SecPolicy
   metadata:
     name: gw-security
     namespace: <cneInstanceNamespace>
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: Gateway
       name: my-gateway
     firewallPolicies:
       - name: my-fw-policy
   

   kubectl apply -f secpolicy.yaml
   

DDoS protection

You can configure DDoS protection with the F5BigDdosGlobal custom resource. Use this resource to set up detection vectors, thresholds, event logging, and allow lists.

apiVersion: k8s.f5net.com/v1
kind: F5BigDdosGlobal
metadata:
  name: ddos-config
  namespace: <cneInstanceNamespace>
spec:
  hslPublisher:
    # Configure HSL publisher for DDoS event logging
  vectors:
    # Configure detection vectors and thresholds
  allowList:
    # IPs exempt from DDoS detection

HSL logging

High-Speed Logging (HSL) sends traffic and security logs to external log collectors. Configure HSL through NetPolicy.

1. Create an HSL publisher.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigLogHslpub
   metadata:
     name: hsl-publisher
     namespace: <cneInstanceNamespace>
   spec:
     # Publisher configuration for remote log server
   

2. Create a log profile.

   apiVersion: k8s.f5net.com/v1
   kind: F5BigLogProfile
   metadata:
     name: my-log-profile
     namespace: <applicationNamespace>
   spec:
     # Log profile referencing HSL publisher
   

Security hardening best practices

Use the following practices to harden your BNK deployment:

• Read-only root filesystems: BNK uses read-only root filesystems for containers. This helps prevent changes to binaries.

• Minimum RBAC permissions: BNK includes minimal cluster-wide RBAC. Review the RBAC justification in Appendix A.

• mTLS for inter-component traffic: All internal traffic uses mutual TLS. Certificates are managed by cert-manager.

• Network policies: Use Kubernetes NetworkPolicy to restrict pod communication.

• Secret management: Store JWT licenses, TLS certificates, and FAR credentials as Kubernetes Secrets. Apply least-privilege RBAC.

• Image pull secrets: Use FAR service accounts with scoped permissions. Rotate keys regularly.

• Audit logging: Enable Kubernetes audit logs to track access to BNK resources.

• DPU isolation: Run only BNK on the DPU node. Do not install other software.

AI inferencing load balancing (early access)

Architecture

The AI Inferencing Load Balancing feature in BNK distributes traffic for Large Language Model (LLM) inference workloads. It uses the NVIDIA LLM Router blueprint and makes routing decisions based on real‑time GPU metrics, queue depth, and model‑specific signals from inference servers.

Core components

Component	Description
TMM with AI LB module	Extends the TMM data plane with LLM‑aware load‑balancing logic.
Metrics Collector	Collects inference metrics such as GPU utilization, queue depth, and token throughput from backend LLM servers.
Policy Engine	Uses built‑in or custom policy scripts to decide how traffic is routed based on collected metrics.
JSON Schema Interface	Provides a standard structure for defining custom load‑balancing algorithms.

Prerequisites

• BNK installed and running on a DPU

• NVIDIA GPU infrastructure with inference servers such as NVIDIA Triton, vLLM, or TGI

• Gateway API routing configured with backends that point to inference endpoints

• AI LB feature flag enabled (Early Access; contact your F5 account team)

Configuration

Configure the AI LB feature using custom resource parameters. These parameters define the metrics endpoint, collection interval, and load‑balancing behavior.

Important

This is an Early Access feature. Schemas, parameters, and supported algorithms may change in future releases. Contact your F5 account team for the latest EA configuration guide and supported platforms.

Configuration options

Option	Description
Built‑in Script	Provides preset algorithms optimized for LLM traffic, such as GPU‑aware round‑robin or least‑tokens‑queued.
Custom Script	Lets you define custom logic using the JSON Schema Interface. You can use your own metrics and routing rules based on model behavior or platform needs.
CR Parameters	Sets metrics endpoint URLs, polling intervals, backend health thresholds, and failover options.

Observability and monitoring

OTEL collector setup

FLO deploys the OTEL Collector. The Collector receives metrics from TMM through the Distributed TODA pipeline and exports them in Prometheus format on port 9090.

# Verify OTEL Collector is running
kubectl get pods --namespace <cneInstanceNamespace> | grep otel

# Check OTEL service
kubectl get svc otel-collector-svc --namespace <cneInstanceNamespace>

Expected ports

Port Name	Port	Purpose
otlp-grpc	4317/TCP	Receives metrics from Observer
otlp-prometheus	9090/TCP	Exports metrics to Prometheus

Prometheus and Grafana integration

Prometheus is not installed by FLO and must be installed manually.

1. Install Prometheus.

   helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
   helm install prometheus prometheus-community/prometheus \
     --namespace monitoring --create-namespace
   

2. Configure Prometheus to scrape OTEL metrics.

   Add this block to your prometheus.yaml scrape configuration:

   scrape_configs:
     - job_name: 'bnk-otel'
       static_configs:
         - targets: ['otel-collector-svc.<cneInstanceNamespace>:9090']
       metric_relabel_configs:
         - source_labels: [__name__]
           regex: '(.*)\-(.*)'
           replacement: 'f5_$1_$2'
           target_label: __name__
           action: replace
   

3. Query BNK metrics.

   Example queries:

   # Virtual server bytes received
   f5_tmm_f5_virtual_server_clientside_received_bytes_total
   
   # Virtual server connections
   f5_tmm_f5_virtual_server_clientside_connections_total
   
   # TMM CPU utilization
   f5_tmm_cpu_utilization
   

   The OTEL exporter adds the f5_tmm_ prefix.

4. Install Grafana and create dashboards.

   helm install grafana grafana/grafana --namespace monitoring
   

   Add Prometheus as a data source in Grafana. Then import BNK dashboards or create your own.

Note

Prometheus and Grafana are not deployed by FLO. Install and configure them manually.

Distributed TODA stats

Distributed TODA processes large volumes of telemetry data through four pod types:

Pod	Role
Receiver (StatefulSet)	Collects metrics from the TMM Scraper and sends them to Observer using gRPC with mTLS.
Observer (StatefulSet)	Aggregates metrics from multiple TMMs and sends them to the OTEL Collector using gRPC with mTLS.
Coordinator (Operator)	Manages collection and aggregation workflows using gRPC with mTLS.
TMM Scraper	Runs inside each TMM pod. Replaces tmstatsd and exposes metrics from tmctl over gRPC.

Note

The metrics export interval is fixed at two minutes in the Coordinator ConfigMap and cannot be changed.

Fluentd logging

Fluentd collects and forwards logs from BNK components. Logs use structured formats and are grouped by component (TMM, Controller, CWC).

QKView and iHealth

Use the CWC QKView API to collect diagnostic data.

# Generate QKView
curl --key cwc_api/client_key.pem \
  --cert cwc_api/client_certificate.pem \
  --cacert cwc_api/ca_certificate.pem \
  https://f5-spk-cwc.f5-cne-core:30881/qkview \
  -H "Authorization: Bearer $CWC_TOKEN"

Upload the resulting file to iHealth for analysis.

High-availability architecture considerations

BNK on DPU uses Kubernetes‑native patterns to provide high availability. The platform maintains traffic flow and control plane stability through the following mechanisms.

Mechanism	Description
TMM DaemonSet redundancy	With multiple DPU nodes, the TMM DaemonSet runs one TMM pod on each node. If a DPU fails, traffic is routed to TMM pods on the remaining nodes.
dSSM Redis Sentinel HA	dSSM uses Redis Sentinel with three replicas. Sentinel promotes a replica if the master fails. Data is stored on persistent volumes.
Control plane resilience	The CNE Controller and CWC run on host nodes. Kubernetes restarts them if a failure occurs.
FLO reconciliation	FLO monitors all BNK components. If a component fails or is deleted, FLO detects the change and redeploys it.
Certificate auto‑renewal	cert‑manager renews TLS certificates before they expire to prevent mTLS issues.

DPU failure handling

When a DPU node fails, BNK and Kubernetes handle recovery through the following sequence:

1. Kubernetes marks the DPU node as NotReady after the node‑monitor‑grace‑period (default: 40 seconds).

2. The TMM pod on the failed DPU moves to Terminating.

3. If your external load balancer or router uses health checks, it stops sending traffic to the failed DPU’s virtual IPs.

4. Other TMM pods on healthy DPU nodes continue processing traffic.

5. FLO detects the change in component status and updates the custom resource conditions.

6. When the DPU recovers, Kubernetes marks the node Ready, the TMM DaemonSet recreates the pod, and FLO reconciles configuration.

Note

BNK provides High Availability Upgrade Phase I capabilities. Full rolling upgrades with zero downtime are planned for future releases.

Appendix A: RBAC minimum privileges

BNK follows the principle of least privilege. The components below require cluster‑wide access and include justification for each permission.

Component	Resource	Privilege	Justification
CNE Controller	nodes	get, list	Needed to create static routes to backend pods for in‑cluster services. Required only when watchNamespace is not empty.
CNE Controller	CRDs	get, list	Determines which F5 CRDs are installed so the controller can start the right watchers.
CNE Controller	namespaces	get, list, watch	Adds or removes OVN annotations. Required only when watchNamespace is not empty.
CNE Controller	webhookconfigs	get, update	Updates the CA bundle for the validation webhook, which is cluster‑scoped.
CWC	nodes	get, list	Gathers node details for telemetry reporting.
CWC	CRDs	get, list	Reads installed F5 CRDs for telemetry and QKView collection.
CWC	namespaces	get, list	Collects namespace details for QKView.

Note

When wholeCluster is set to true (daemon-set mode) and watchNamespace is empty (default), the CNE Controller does not need permissions for nodes, namespaces, or adminpolicybasedexternalroutes.

Appendix B: Migration from SPK to BNK

The following guidance applies when migrating from F5 SPK (Service Proxy for Kubernetes) to BNK (BIG‑IP Next for Kubernetes).

Key differences

Aspect	SPK (Previous)	BNK
Product Name	Service Proxy for Kubernetes (SPK)	BIG‑IP Next for Kubernetes (BNK)
Target Platform	Kubernetes on standard servers	Kubernetes on NVIDIA BlueField‑3 DPU
Lifecycle Management	Manual Helm‑based workflows	Automated lifecycle management through FLO
CRD Management	Manual installs	FLO installs CRDs and validates versions
Environment Detection	Manual node labels	Env‑Discovery auto‑detects DPUs and configuration
Licensing	Connected or Disconnected	Connected or FLP (License Proxy)
Gateway API	v1.0 (limited)	v1.2.0 with SecPolicy and NetPolicy
Observability	OTEL + tmstatsd	Distributed TODA + OTEL (replaces tmstatsd)
Security Extensions	Basic firewall CRs	SecPolicy for Gateway‑level firewall and DDoS
iRules	Not supported in Gateway API	NetPolicy adds iRules support in Gateway API

Migration steps

1. Export all SPK CRs (F5SPKVlan, F5SPKEgress, F5SPKSnatpool, L4Route, and others) from the current cluster.

2. SPK networking CRs (F5SPKVlan, F5SPKStaticRoute, and others) are compatible with BNK and do not need format changes.

3. Uninstall SPK following the SPK documentation.

4. Prepare the DPU infrastructure. See Infrastructure Preparation.

5. Install BNK using FLO. See BNK Installation.

6. Reapply exported CRs. Networking CRs should apply without changes.

7. Convert any Ingress‑based configuration to Gateway API resources such as HTTPRoute or GRPCRoute.

8. Configure new BNK capabilities such as FLP licensing, SecPolicy, NetPolicy, and Distributed TODA.

9. Validate all traffic paths and services after migration.

Important

SPK and BNK cannot run in the same cluster. Schedule downtime and plan a maintenance window for migration.