TMM Resources

Overview

BIG-IP Next for Kubernetes uses standard Kubernetes Requests and Limits parameters to manage container CPU and memory resources. If you intend to modify the Traffic Management Microkernel (TMM) resource allocations, it is important to understand how Requests and Limits are applied to ensure the TMM Pod runs in Guaranteed QoS.

This document describes the default Requests and Limits values, and demonstrates how to properly modify the default values.

TMM Pod limit values

The containers in the TMM Pod install with the following default resources.limits:

Container memory cpu hugepages-2Mi
f5-tmm 2Gi 1 3Gi
blobd 4Gi 1 None
debug 1Gi “500m” None
f5-tmm-routing 1Gi “700m” None
f5-tmm-routed 512Mi “300m” None

f5-TMM Pod value

Replace performance-profile with the name of the Performance-profile on your cluster.

# f5-tmm Pod
tmm:
  pod:
    annotations:
      # Performance optimization per-pod.
      cpu-load-balancing.crio.io: disable
      cpu-quota.crio.io: disable
      irq-load-balancing.crio.io: disable
  runtimeClassName: performance-performance-profile

Guaranteed QoS class

The TMM container must run in the Guaranteed QoS class; top-priority Pods that are guaranteed to only be killed when exceeding their configured limits. To run as Guaranteed QoS class, the Pod resources.limits and resources.requests parameters must specify the same values. By default, the TMM Pod’s resources.limits are set to the following values:

Note: When the resources.requests parameter is omitted from the Helm values file, it inherits the resources.limits values.
tmm:
  resources:
    limits:
      cpu: "2"
      hugepages-2Mi: "3Gi"
      memory: "2Gi"
Important: Memory values must be set using either the Mi or Gi suffixes. Do not use full byte values such as 1048576, or the G and M suffixes. Also, do not allocate CPU cores using fractional numbers. These values will cause the TMM Pod to run in either BestEffort or Burstable QoS class.

Verify the QoS class

The TMM Pod’s QoS class can be determined by running the following command:

kubectl get pod -l app=f5-tmm -o jsonpath='{..qosClass}{"\n"}' -n <project>

In this example, the TMM Pod is in the default Project:

kubectl get pod -l app=f5-tmm -o jsonpath='{..qosClass}{"\n"}' -n default

Guaranteed

Modifying defaults

The TMM requires hugepages to enable direct memory access (DMS). When allocating additional TMM CPU cores, hugepages must be pre-allocated using the hugepages-2Mi parameter. To calculate the minimum amount of hugepages, use the following formula: 1.5GB x TMM CPU count. For example, allocating 4 TMM CPUs requires minimum 6GB of hugepages memory. To allocate 4 TMM CPU cores to the f5-tmm container, add the following limits to the BIG-IP Next for Kubernetes SPKInstance CR

tmm:
  resources:
    limits:
      cpu: "4"
      hugepages-2Mi: "6Gi"
      memory: "2Gi"
  palCPUSet: "0-3" 
  tmmMapresHugepages: 3072
Note: To calculate the minimum tmmMapresHugepages use the following formula: 768 x TMM CPU count

For example: allocating 4 TMM CPUs requires minimum 3072 hugepages. For example: allocating 4 TMM CPUs update the palCPUSet to 0-3 (The ranges of CPU identifiers that are to be dedicated to TMM instances)