Configure sidecar listeners for exact balancing

Introduction

You can increase the throughput of an Envoy sidecar proxy by configuring its listener for exact balancing so that TCP connections are nearly evenly distributed between worker threads.

Prerequisite

To use exact balancing, you must be running Aspen Mesh 1.11.5-am1 or later.

When to use exact balancing

Use exact balancing when there are a small number of connections that rarely cycle (for example, service-mesh gRPC egress). Exact balancing may also be beneficial in other situations, but you should verify that using it provides the throughput you expect. For ideas on how to test throughput and information on how to enable sidecar connection-level metrics, see the articles Sailing Faster with Istio, Part 1 and Sailing Faster with Istio, Part 2.

Configure sidecar listeners for exact balancing

1. Use the following template to create inbound-filter manifests that cover every pod with a sidecar (regardless of whether it acts only as a server) whose listener you want to use exact balancing.

   Because the inbound-filter manifest applies only to pods with sidecars in the specified namespace and that have the optional specified label, you may need to create more than one inbound-filter manifest.

   apiVersion: networking.istio.io/v1alpha3
   kind: EnvoyFilter
   metadata:
     name: <inboundFilterName>
     namespace: <workloadNamespaceName>
   spec:
     workloadSelector:  # Limit the filter to pods with the specified label (optional).
       labels:
         app: <appLabel>
     configPatches:
       - applyTo: LISTENER
         match:
           context: SIDECAR_INBOUND
           listener:
             portNumber: 15006
         patch:
           operation: MERGE
           value:
             connection_balance_config:
               exact_balance: {}
   

2. Use the following template to create outbound-filter manifests that cover every pod with a sidecar that acts as a client and whose listener you want to use exact balancing.

   Because the outbound-filter manifest applies only to pods that have sidecars (and that act as clients that are communicating with servers on the specified ports) in the specified namespace and that have the optional specified label, you may need to create more than one outbound-filter manifest.

   Note

   You must specify at least one port number so the filter doesn’t apply to all ports (and therefore any reserved Istio ports).

   apiVersion: networking.istio.io/v1alpha3
   kind: EnvoyFilter
   metadata:
     name: <outboundFilterName>
     namespace: <clientWorkloadNamespaceName>
   spec:
     workloadSelector:  # Limit the filter to pods with the specified label (optional).
       labels:
         app: <clientAppLabel>
     configPatches:
       - applyTo: LISTENER
         match:
           context: SIDECAR_OUTBOUND
           listener:
             portNumber: <serverServicePortNumber1>
         patch:
           operation: MERGE
           value:
             connection_balance_config:
               exact_balance: {}
       ...
       - applyTo: LISTENER
         match:
           context: SIDECAR_OUTBOUND
           listener:
             portNumber: <serverServicePortNumberN>
         patch:
           operation: MERGE
           value:
             connection_balance_config:
               exact_balance: {}
   

3. For each filter manifest, create the filter:

   $ kubectl create -f <filterManifestFilename>
   

4. Restart all pods whose sidecars should be using a filter.

   If you don’t restart a pod, its sidecar’s listener won’t use exact balancing.

5. For each pod whose sidecar should be using a filter, verify that its sidecar’s listener is configured for exact balancing:

   $ istioctl proxy-config listeners --port <portNumber> -o json  <podName>.<namespaceName> | grep -A2 connectionBalance
   

   Expected output:

   "connectionBalanceConfig": {
       "exactBalance": {}
   },