Configure DOCA Offloads on DPU¶
Overview¶
BIG-IP Next for Kubernetes (BNK) uses the NVIDIA BlueField-3 Data Processing Unit (DPU) to provide hardware acceleration through Data Center Operations and Communications Acceleration (DOCA) APIs. These APIs deliver secure, high-speed encryption, decryption, and traffic filtering capabilities. This guide shows you how to turn on and control these acceleration features with BNK’s built-in resources.
Available acceleration types¶
The BlueField-3 DPU supports four acceleration types. The table below summarizes each type, its use cases, and its benefits.
Acceleration type |
Description |
Use cases |
Benefits |
|---|---|---|---|
Crypto hardware |
Uses the AES-GCM engine to improve performance and security for traffic management operations |
- High-throughput TLS/SSL processing |
- Reduced CPU utilization for cryptographic operations |
Firewall/ACL hardware |
Uses DPU hardware acceleration to optimize traffic management at the network layer through Access Control Lists (ACLs) |
- DDoS protection at line rate |
- DPU processes first packet and enforces drop rules without host CPU |
Global DDoS hardware |
Provides hardware-accelerated traffic inspection and mitigation to detect and block distributed denial-of-service attacks at line rate |
- Multi-vector attack mitigation |
- Line-rate inspection without CPU involvement |
Flow offload |
Automatically offloads established TCP and UDP connection flows to hardware after initial flow establishment |
- High-throughput applications |
- Wire-speed packet processing for established flows |
Prerequisites¶
Verify that all TMM pods are deployed on DPU nodes. This should happen automatically if the system is set up correctly.
kubectl get pods -o wide -l app=f5-tmm
# expected output
...
f5-tmm-6dmsm 7/7 Running 0 dpu-1 6d7h
f5-tmm-6njxv 7/7 Running 0 dpu-2 6d7h
Turn on hardware acceleration¶
1. Create the F5BigGlobalOptions resource¶
The F5BigGlobalOptions CRD lets you define multiple behaviors at the global level (Traffic Management Microkernel - TMM) on the BlueField-3 DPU.
Create a file named accel-on-global-options.yaml and turn on the acceleration toggles. Here’s an example:
apiVersion: k8s.f5net.com/v1
kind: F5BigGlobalOptions
metadata:
name: global-options
spec:
# Crypto Hardware Acceleration
crypto:
hwAccel: true # ← Turn on crypto hardware acceleration (default: true)
# Firewall Hardware Offload
firewall:
hardwareOffload: true # ← Turn on firewall rule offloading
defaultAction: drop # ← Options: accept, reject, drop, ""
defaultActionLog: true # ← Log when default action is applied
2. Apply the configuration¶
kubectl apply -f global-options.yaml
3. Verify the configuration¶
Check that the resource was created:
kubectl get f5bigglobaloptions -n app-namespace
Set up accelerated firewall ACL rules¶
Once firewall acceleration is turned on in the global options and applied, you can apply specific firewall rules to manage traffic access.
Tip
Understand firewall ACL offload behavior
Only drop actions are accelerated. Other action types still require TMM software to evaluate the condition
First packet(s) of any flow are processed in software to determine eligibility for acceleration
Hardware offload entries are automatically cleaned up after an idle period (two seconds)
Tip
Understand the firewall ACL configuration chain
BNK offers ACL offload to resources like Gateways and HTTP routes. This works by chaining together a set of resources to link these application layer constructs to the hardware.
The chaining uses the name of associated resources as references.
The relationship looks like:
[HTTP Route] - serviced by -> [Gateway] <- secured by -> [SecPolicy] -secured by-> [F5BigFwPolicy] -defines-> [F5BigFwRulelist]
Note
In the sections below, we describe creating named firewall rule lists that are referenced from firewall policies. You can also create firewall rules directly in firewall policies.
1. Create the firewall rule list CR¶
The example below shows how to grant access from a particular subnet to a particular destination.
Create a file with a name of your choosing (for example, fw-rule-set-accept-subnet.yaml) and paste the content below:
apiVersion: "k8s.f5net.com/v1"
kind: F5BigFwRulelist
metadata:
name: "allow-subnet-11"
namespace: f5-bnk-instance # controller namespace where TMM and F5ingress are deployed
spec:
rule:
- name: rule1
ipProtocol: tcp
source:
addresses:
- 192.168.10.112/32 # ← (1)
ports:
- "80" # ← (2)
# vlans:
# - "name-of-vlan-cr" # ← (3)
destination:
addresses:
- "22.55.44.44" # ← (4)
ports:
- "80" # ← (5)
logging: true
action: drop
# servicePolicy: "name-of-service-policy1-cr" # ← (6)
Where the numbered entries indicate:
Source subnet. This rule applies to packets whose source address matches the subnet. The
addressesfield accepts a variable number of entriesSource port. You typically don’t need to specify this unless the source uses a well-known or fixed port number. The
portsfield accepts a variable number of entriesSource VLAN. If specified, this rule applies to packets arriving on the specified VLAN. The
vlansfield accepts the names of the VLAN CRs appliedDestination subnet. This rule applies to packets whose destination address matches the subnet. The
addressesfield accepts a variable number of entriesDestination port. If specified, this rule applies to packets whose destination port matches the entry. The
portsfield accepts a variable number of entriesService Policy. The
servicePolicyfield accepts the name of the servicePolicy CR applied
2. Create the firewall policy CR¶
The example below shows how to associate a firewall policy with the rule list above.
Create a file with a name of your choosing (for example, fw-policy-accept-subnet.yaml) and paste the content below:
apiVersion: "k8s.f5net.com/v1"
kind: F5BigFwPolicy
metadata:
name: fw-pol-accept-subnet
namespace: f5-bnk-instance # controller namespace where TMM and F5ingress are deployed
spec:
rule:
- name: "allow-subnet-11"
ruleList: "allow-subnet-11"
3. Create the SecPolicy CR¶
The example below shows how to associate a SecPolicy with the firewall policy above.
Create a file with a name of your choosing (for example, bnk-sec-policy-accept-subnet.yaml) and paste the content below:
apiVersion: "gateway.k8s.f5.com/v1alpha1"
kind: SecPolicy
metadata:
name: sec-pol-accept-subnet
namespace: f5-bnk-instance # controller namespace where TMM and CNEController are deployed
spec:
extensionRefs:
- name: fw-pol-accept-subnet
group: k8s.f5net.com
kind: F5BigFwPolicy
targetRefs:
- group: gateway.networking.k8s.io
kind: GatewayClass
name: f5-gatewayclass
4. Apply the CRs¶
kubectl apply -f fw-policy-accept-subnet.yaml
kubectl apply -f bnk-sec-policy-accept-subnet.yaml
Validation best practices¶
Start with monitoring
Turn on
defaultActionLog: trueinitially to understand traffic patterns
Optimize rule order - Place high-volume drop rules with lower priority to maximize offload
Monitor regularly - Watch HSL logs for first-packet processing patterns
Test incrementally - Turn on one acceleration type at a time
Document changes - Keep track of configuration changes for troubleshooting
Set up global DDoS protection¶
Once the system prerequisites are in place, you can set up global DDoS protection to detect and mitigate volumetric attacks at the hardware level.
Tip
Understand global DDoS offload behavior
DDoS vector detection operates on per-flow statistics collected by the DPU hardware
When a configured threshold is exceeded, the DPU enforces rate limiting at line rate without CPU involvement
You can set up an allow list to exempt trusted sources from DDoS mitigation
DDoS statistics are reflected in TMM stat tables and can be monitored with
tmctl
Tip
Understand the global DDoS configuration chain
BNK offers global DDoS protection that binds to a GatewayClass through a SecPolicy. This works by chaining together a set of resources to link the DDoS configuration to the network infrastructure. The chaining uses the name of associated resources as references.
The relationship looks like:
[GatewayClass] <- secured by -> [SecPolicy] -references-> [F5BigDdosGlobal] -references-> [F5BigCneAddresslist]
Prerequisites¶
Verify the following resources are deployed before you set up global DDoS protection:
# Verify TMM pods are running on DPU nodes
kubectl get pods -o wide -l app=f5-tmm
# Verify GatewayClass exists
kubectl get gatewayclass f5-gatewayclass
# Verify Gateway is deployed
kubectl get gateways -A
# Verify virtual servers are active (through Gateway API routes)
kubectl get httproutes -A
1. Create the global DDoS CR¶
The example below shows a global DDoS policy that mitigates ACK flood, SYN flood, and UDP flood attacks. It includes configurable thresholds and rate limits, along with an allow list for trusted sources.
Create a file with a name of your choosing (for example, ddos-global-with-allowlist.yaml) and paste the content below:
apiVersion: "k8s.f5net.com/v1"
kind: F5BigDdosGlobal
metadata:
name: ddos-with-allowlist
spec:
vectors:
ackFlood:
state: "mitigation"
detectionThresholdEps: 500 # ← (1)
rateLimit: 500 # ← (2)
synFlood:
state: "mitigation"
detectionThresholdEps: 500 # ← (1)
rateLimit: 400 # ← (2)
udpFlood:
state: "mitigation"
detectionThresholdEps: 500 # ← (1)
rateLimit: 100 # ← (2)
allowList:
srcAddressList: "allow-addr-list" # ← (3)
entries: # ← (4)
- name: tcp-v6-all
ipProtocol: "tcp"
entryType: "v6-all"
- name: icmp-v4-all
ipProtocol: "icmp"
entryType: "v4-all"
Where the numbered entries indicate:
Detection threshold. Events per second (EPS) threshold at which an attack is registered for the vector. Adjust this based on your expected traffic baselines
Rate limit. The rate (in EPS) to which traffic is limited when an attack is detected for the vector
Source address list. The name of the
F5BigCneAddresslistCR that defines trusted source addresses exempt from DDoS mitigationRich allow list entries. Up to eight entries that define protocol and address-family combinations exempt from DDoS mitigation. The
entryTypefield accepts values such asv4-allandv6-all
2. Create the address list CR¶
The example below shows an address list of trusted sources that are exempt from DDoS rate limiting.
Create a file with a name of your choosing (for example, ddos-allow-addr-list.yaml) and paste the content below:
apiVersion: "k8s.f5net.com/v1"
kind: F5BigCneAddresslist
metadata:
name: allow-addr-list # ← (1)
spec:
addresses: # ← (2)
- 192.168.30.160
- 192.168.30.161
- 192.168.110.112
Where the numbered entries indicate:
Address list name. This must match the
srcAddressListvalue referenced in the global DDoS CRAddresses. Trusted source IP addresses that bypass the DDoS filter. The
addressesfield accepts a variable number of entries
3. Create the SecPolicy CR¶
The example below shows how to associate a SecPolicy with the global DDoS policy above.
Create a file with a name of your choosing (for example, bnk-sec-policy-ddos.yaml) and paste the content below:
apiVersion: gateway.k8s.f5.com/v1alpha1
kind: SecPolicy
metadata:
name: secpolicy
namespace: f5-bnk-instance # ← (1)
spec:
extensionRefs:
- group: k8s.f5net.com
kind: F5BigDdosGlobal
name: ddos-with-allowlist # ← (2)
targetRefs:
- group: gateway.networking.k8s.io
kind: GatewayClass
name: f5-gatewayclass # ← (3)
Where the numbered entries indicate:
Namespace. The controller namespace where TMM and the CNE controller are deployed
DDoS policy name. This must match the name of the
F5BigDdosGlobalCR you appliedGatewayClass name. The target GatewayClass that the DDoS policy binds to
4. Apply the CRs¶
kubectl apply -f ddos-global-with-allowlist.yaml
kubectl apply -f ddos-allow-addr-list.yaml
kubectl apply -f bnk-sec-policy-ddos.yaml
5. Verify the configuration¶
Check F5ingress controller logs¶
When you apply a CR, the F5ingress controller picks up the configuration and sends an AsyncAPI (gRPC) message to all TMM pods. Verify that the configuration was received:
# Get the F5ingress controller pod name
kubectl get pods -l app=f5-cne-controller -n f5-bnk-instance
# View the controller logs
kubectl logs <F5INGRESS-POD-NAME> -n f5-bnk-instance
Note
You can use the -f flag to tail logs in real time, for example:
kubectl logs -f <F5INGRESS-POD-NAME> -n f5-bnk-instance
Check TMM logs¶
After the F5ingress controller dispatches the configuration, all TMM pods should receive and apply it. Verify by checking the TMM logs:
# Get TMM pod names
kubectl get pods -l app=f5-tmm -n f5-bnk-instance
# View the TMM logs (default container: f5-tmm)
kubectl logs <TMM-POD-NAME> -n f5-bnk-instance
Note
TMM pods contain multiple containers, each with separate logs.
Use the -c flag to specify a container, for example:
kubectl logs -f <TMM-POD-NAME> -c debug -n f5-bnk-instance
Verify DDoS statistics in TMM¶
Access the TMM debug container and inspect the DDoS stat table. Confirm that the global DDoS vectors are populated:
# Access the debug container of a TMM pod
kubectl exec -it <TMM-POD-NAME> -c debug -n f5-bnk-instance -- bash
# Inside the container, view the DDoS stat table
tmctl -w 150 -d /var/tmstat/blade/ dos_stat
If the global DDoS CR is applied correctly, the stat table displays your configured vectors with active statistics. An empty table means the configuration hasn’t been applied.
Validation best practices¶
Start with monitoring
Set conservative detection thresholds initially and observe traffic patterns before tightening limits
Baseline your traffic - Understand normal EPS rates for each vector type before you set detection thresholds
Test the allow list - Verify that trusted sources in the address list aren’t rate-limited during an active mitigation event
Review stat tables regularly - Monitor
dos_statthroughtmctlto confirm vector states and attack detection countsDocument changes - Keep track of threshold and rate-limit changes for troubleshooting
Set up flow offload¶
Flow offload moves established TCP and UDP connections into hardware on the BlueField-3 DPU. After the initial flow is established, subsequent packets bypass software processing entirely.
Tip
Understand flow offload behavior
The initial packet (SYN for TCP) is processed in software by TMM to establish the connection and apply policies
Once the connection is established, a hardware flow entry is created in the DPU using 6-tuple matching (source IP, destination IP, source port, destination port, protocol, and VLAN)
Subsequent packets matching the flow are processed entirely in hardware at line rate without CPU involvement
Idle flows are automatically removed from hardware after a timeout period to free resources
Tip
Understand the flow offload configuration chain
Flow offload is turned on through L4Route resources bound to a Gateway. The DPU automatically offloads eligible flows during normal operation once the route is active. The relationship looks like:
[L4Route] - attached to -> [Gateway] - managed by -> [GatewayClass]
Note
Flow offload is turned on by default when you create an L4Route CR. No additional global option toggles are required.
Prerequisites¶
Verify the following resources are deployed before you set up flow offload:
# Verify TMM pods are running on DPU nodes
kubectl get pods -o wide -l app=f5-tmm
# Verify GatewayClass exists
kubectl get gatewayclass f5-gatewayclass
# Verify Gateway is deployed
kubectl get gateways -A
1. Create the L4Route CR¶
The example below shows an L4 TCP route that directs traffic from a Gateway listener to a backend service. The DPU automatically offloads eligible flows once this route is active.
Create a file with a name of your choosing (for example, l4-tcp-route.yaml) and paste the content below:
apiVersion: gateway.k8s.f5.com/v1
kind: L4Route
metadata:
name: l4-tcp-app
namespace: bnk-app-1 # ← (1)
spec:
protocol: TCP
parentRefs:
- name: f5-gateway-tcp # ← (2)
namespace: bnk-app-1 # ← (3)
port: 80
sectionName: l4-tcp # ← (4)
rules:
- backendRefs:
- name: nginx-service-1 # ← (5)
port: 8080 # ← (6)
#pvaAccelerationMode: full/assisted # ← (7)
Where the numbered entries indicate:
Namespace. Must be the same namespace as the Gateway resource
Gateway name. The name of the Gateway resource this route attaches to
Gateway namespace. Must match the namespace of the referenced Gateway
Section name. The name of the listener defined in the Gateway CR that this route binds to
Backend service name. The name of the Kubernetes service that receives traffic
Backend service port. The port on which the backend service listens
PVA acceleration mode. Optional. Controls the acceleration behavior. Defaults to
full. Accepted values arefullandassisted
2. Add L4Route listener to the Gateway CR¶
The Gateway CR must include a listener that the L4Route can bind to. The example below shows a Gateway with a TCP listener set up for L4 routing.
Create a file with a name of your choosing (for example, gateway-with-l4-listener.yaml) and paste the content below:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: f5-gateway-tcp
namespace: bnk-app-1 # ← (1)
spec:
infrastructure:
addresses:
- type: IPAddress
value: 192.168.110.119 # ← (2)
gatewayClassName: f5-gatewayclass # ← (3)
listeners:
- name: l4-tcp # ← (4)
protocol: TCP
port: 80 # ← (5)
allowedRoutes:
namespaces:
from: Same
kinds:
- group: gateway.k8s.f5.com/v1
kind: L4Route # ← (6)
Where the numbered entries indicate:
Namespace. The application namespace where the Gateway and L4Route are deployed
Virtual server IP. The IP address the virtual server listens on for incoming traffic
GatewayClass name. The GatewayClass that manages this Gateway
Listener name. The name referenced by the L4Route’s
sectionNamefieldListener port. The port on which the virtual server accepts incoming connections
Route kind. Restricts this listener to accept only
L4Routeresources
3. Apply the CRs¶
kubectl apply -f gateway-with-l4-listener.yaml
kubectl apply -f l4-tcp-route.yaml
4. Verify the configuration¶
Access the TMM debug container¶
# Get TMM pod names
kubectl get pods -l app=f5-tmm -n f5-bnk-instance
# Access the debug container of a TMM pod
kubectl exec -it <TMM-POD-NAME> -c debug -n f5-bnk-instance -- bash
Check flow entry statistics (before traffic)¶
Inside the debug container, inspect the flow offload stat table to establish a baseline:
tmctl -w 150 -d /var/tmstat/blade/ doca_flow_entries
Expected output (no traffic yet):
port_id type_id type queue request offloaded deleted total aged fails
------- ------- -------- ----- ------- --------- ------- ----- ---- -----
0 0 STATIC 0 0 29 0 29 0 0
0 1 DDOS 0 0 4 0 4 0 0
0 2 DDOS RWL 0 0 0 0 0 0 0
0 3 DDOS WL 0 0 0 0 0 0 0
0 4 TUNNEL 0 0 0 0 0 0 0
0 5 FLOW 0 0 0 0 0 0 0 # ← Flow stats (client side)
0 6 ACL 0 0 0 0 0 0 0
1 0 STATIC 0 0 29 0 29 0 0
1 1 DDOS 0 0 4 0 4 0 0
1 2 DDOS RWL 0 0 0 0 0 0 0
1 3 DDOS WL 0 0 0 0 0 0 0
1 4 TUNNEL 0 0 0 0 0 0 0
1 5 FLOW 0 0 0 0 0 0 0 # ← Flow stats (server side)
1 6 ACL 0 0 0 0 0 0 0
Note
Port 0 represents the client side. Port 1 represents the server side.
Send traffic to the virtual server¶
From an external client with connectivity to the DPU and TMM, send traffic to the virtual server:
for i in {1..100}; do
curl 192.168.110.119:80 # ← Virtual server VIP and port
done
Verify flow entries are offloaded¶
Re-inspect the flow offload stat table inside the debug container:
tmctl -w 150 -d /var/tmstat/blade/ doca_flow_entries
Expected output (after traffic):
port_id type_id type queue request offloaded deleted total aged fails
------- ------- -------- ----- ------- --------- ------- ----- ---- -----
0 0 STATIC 0 0 29 0 29 0 0
0 1 DDOS 0 0 4 0 4 0 0
0 2 DDOS RWL 0 0 0 0 0 0 0
0 3 DDOS WL 0 0 0 0 0 0 0
0 4 TUNNEL 0 0 0 0 0 0 0
0 5 FLOW 0 0 0 123 123 0 0
0 6 ACL 0 0 0 0 0 0 0
1 0 STATIC 0 0 29 0 29 0 0
1 1 DDOS 0 0 4 0 4 0 0
1 2 DDOS RWL 0 0 0 0 0 0 0
1 3 DDOS WL 0 0 0 0 0 0 0
1 4 TUNNEL 0 0 0 0 0 0 0
1 5 FLOW 0 0 0 123 123 0 0
1 6 ACL 0 0 0 0 0 0 0
Note
The offloaded column shows the number of flows currently offloaded to hardware. This value resets to 0 when flows are deleted after the idle timeout.
Validation best practices¶
Start with a small traffic sample
Send a known number of requests and verify that the
totalandoffloadedcounts match expectations
Monitor both ports - Confirm flow entries appear on both
port 0(client side) andport 1(server side) for bi-directional offloadCheck for failures - A non-zero
failscolumn means the DPU couldn’t offload certain flows. Investigate DPU resource utilizationTest idle timeout behavior - Stop sending traffic and observe that
offloadedcounts return to0as idle flows are cleaned upDocument changes - Keep track of L4Route and Gateway changes for troubleshooting