Customize resource requests and limits#

About customizing resource requests and limits#

Introduction#

Kubernetes uses resource requests and limits—including those for CPU, ephemeral storage, and memory—to control the scheduling and eviction of pods.

Pod resource requests and limits are derived from containers#

You specify resource requests and limits for containers, but Kubernetes makes most scheduling and eviction decisions based on the resource requests and limits for a pod. A pod’s resource request or limit is the sum of the resource requests or limits of that type for each container in the pod.

About customizing sidecar resource requests and limits#

You can customize resource requests and limits for Istio-proxy (Envoy) sidecar containers globally or per pod. Customization affects both manually and automatically injected sidecars.

Methods for customizing sidecar resource requests and limits#

There are two methods you can use to customize a sidecar resource request or limit:

Method	Description
Global	The value you specify applies to all sidecars and acts as an override to the default global value, if any, in the sidecar injection template.
Per pod	The value you specify applies to only the sidecar of the specified pod and acts as an override to the custom global value (if specified) and the default global value, if any, in the sidecar injection template.

About specifying non-sidecar resource requests and limits#

You can specify resource requests and limits for non-sidecar containers only per pod. Learn how to specify resource requests and limits for non-sidecar containers.

Important guidelines#

Warning!#

To avoid unintended pod scheduling and eviction problems, carefully read and follow the guidelines below when specifying resource requests and limits for both sidecar and non-sidecar containers.

Appropriate-value guideline#

When specifying resource requests and limits, choose appropriate values that aren’t lower or higher than necessary.

Important

Before using the global method to customize a sidecar resource request or limit, use the per-pod method on as many pods as possible to test whether the value is appropriate.

Every-container guideline#

If you specify a resource request or limit, it’s highly recommended that you do so for every container in a pod. This ensures that Kubernetes has the information necessary to calculate the appropriate resource request or limit for the pod.

Consequences of violating the guidelines#

The following tables describe, for each type of resource, what can happen when you don’t follow the guidelines.

CPU#

If a pod’s CPU	Then
Request is lower than necessary	On a node whose CPU usage is near capacity, Kubernetes will allocate less CPU time to the pod relative to other pods whose CPU requests are higher.
Request is higher than necessary	Kubernetes is less likely to be able to schedule the pod.
Limit is lower than necessary	The pod’s CPU usage will exceed the limit, and Kubernetes is more likely to throttle the pod’s CPU usage.
Limit is higher than necessary	The performance of other pods on the node may be negatively impacted.

Ephemeral storage#

If a pod’s ephemeral-storage	Then
Request is lower than necessary	If the pod’s ephemeral-storage usage exceeds the request, Kubernetes is more likely to evict the pod if its node runs low on ephemeral storage.
Request is higher than necessary	Kubernetes is less likely to be able to schedule the pod.
Limit is lower than necessary	The pod’s ephemeral-storage usage will exceed the limit, and Kubernetes will evict the pod, regardless of the node’s ephemeral-storage availability.
Limit is higher than necessary	The ephemeral storage available to other pods on the node may be negatively impacted.

Memory#

If a pod’s memory	Then
Request is lower than necessary	If a container in the pod exceeds its memory request, Kubernetes is more likely to evict the pod if its node runs low on memory.
Request is higher than necessary	Kubernetes is less likely to be able to schedule the pod.
Limit is lower than necessary	The pod’s memory usage will exceed the limit, and Kubernetes will evict the pod, regardless of the node’s memory availability.
Limit is higher than necessary	The memory available to other pods on the node may be negatively impacted.

Customize sidecar resource requests and limits#

Introduction#

To decide whether to customize sidecar resource requests and limits, you can first determine their default global values.

Determine the default global sidecar CPU and memory requests and limits#

Note

For ephemeral storage, there is no default global sidecar request or limit.

Change to the Aspen Mesh release directory.

Change to the directory for the istio-discovery chart:

$ cd manifests/charts/istio-control/istio-discovery

In the values.yaml file, look for the following fields to determine the default global sidecar CPU and memory requests and limits:
- CPU request: .global.proxy.resources.requests.cpu
- CPU limit: .global.proxy.resources.limits.cpu
- Memory request: .global.proxy.resources.requests.memory
- Memory limit: .global.proxy.resources.limits.memory

Customize the global sidecar CPU and memory requests and limits#

Important

For ephemeral storage, don’t specify a global sidecar request or limit (note that there is no default global sidecar request or limit either). Specify only per-pod sidecar requests and limits for ephemeral storage.

Open your Aspen Mesh override values file (aspen-mesh-override-values.yaml) in a text editor.

Copy the following information and paste it into the file:

# Customize global sidecar resource requests and limits.
# Important - For ephemeral storage, don't specify a global sidecar request
# or limit. Specify only per-pod sidecar requests and limits for ephemeral
# storage.
global:
  proxy:
    resources:
      requests:
        cpu: 
        memory: 
      limits:
        cpu: 
        memory: 

Delete the fields for any resource requests or limits that you don’t want to customize.
Set the values for the remaining fields (for example, 120m for .proxy.resources.requests.cpu).
Save and close the file.
Perform a clean installation or an upgrade of Aspen Mesh.

Customize per-pod sidecar resource requests and limits#

Open the pod’s manifest.

Copy the following information and paste it into the file in the .spec.containers[] list:

  # Customize per-pod sidecar resource requests and limits.
  - name: istio-proxy
    image: auto
    resources:
      requests:
        cpu: 
        ephemeral-storage: 
        memory: 
      limits:
        cpu: 
        ephemeral-storage: 
        memory: 

Delete the fields for any resource requests or limits that you don’t want to customize.
Set the values for the remaining fields (for example, 50Mi for .proxy.resources.requests.ephemeral-storage).
Save and close the file.

Is the pod already running?

Yes:

Restart the pod:

$ kubectl delete pod <podName> --namespace <namespaceName>

No:

Create the pod:

$ kubectl create pod -f <podManifestFilename> --namespace <namespaceName>