6.2. Packet Flow in an SSL Orchestrator Reverse Proxy

6.2.1. What it is

If you have a basic understanding of the outbound packet flow through SSL Orchestrator, as described above, the same knowledge can be applied to the inbound use case. Inbound topologies essentially differentiate from outbound in the way certificates are handled. For inbound topologies, it is expected that the BIG-IP possesses the server certificates and private keys.

That said, there are three inbound topology modes important to describe further. The layer 2 inbound mode ONLY differentiates from the outbound layer 2 mode in the way it handles certificates. For all of the inbound topology modes, packet flow through the service chain is the same as described above.

6.2.2. Inbound Gateway Mode

When you define an inbound layer 3 topology, you have two options depending on how traffic flows to or through the topology. In the Gateway mode, the topology becomes a routing point. The destination IP address in the client’s packet is an address behind the BIG-IP, thus the topology contains a wildcard listener (ex. and forwards (routes) the traffic inbound to a separate destination, load balancer, or next hop route. No NAT is applied to the traffic, though SNAT can be applied as required. In the Gateway mode implementation, the corresponding LTM virtual server has a wildcard destination address (, address and port translation disabled, and no pool assignment. Note that while the destination IP address is behind the BIG-IP in this mode, as in any routing configuration the destination MAC address is on the BIG-IP.

6.2.3. Inbound Application Mode

In Application Mode, the topology is a termination point. In this case, the destination address in the client’s packet is an IP address assigned to the topology listener. In the Application mode implementation, the corresponding LTM virtual server defines the target destination IP address and port, address and port translation are enabled, a pool is assigned, and NAT is performed to the load balanced pool members. Note here that the destination IP and destination MAC address are on the BIG-IP.

6.2.4. Existing Application Mode

The Existing Application mode enables you to attach the SSL Orchestrator security directly to an existing LTM reverse proxy virtual server. This is most useful in situations where a BIG-IP LTM is already deployed and handling application traffic. The SSL Orchestrator security policy can be attached to provide dynamic decrypted service chaining to a set of security devices. It is expected that the existing LTM virtual server already handles its own SSL decryption (client/server SSL profiles) and traffic management (pool). The Existing Application topology workflow simply contains configuration steps for defining security services, service chains, and a security policy. The resulting security policy manifests as an Access per-request policy.


Mutual TLS (mTLS), where the client presents a certificate to the server for authentication, has specific implications in a reverse proxy environment.

In an mTLS handshake, the server will request a certificate from the client. The client will then generate two artifacts. The first artifact will be the certificate itself, passed back to the server. Following that, the client will digitally sign a piece of information (a portion of the handshake log), and pass that to the server in a second message. A digital signature is essentially a hash value encrypted with the sender’s private key.

For any device between the client and server to be able to decrypt this traffic, it must be able to produce the same digital signature - using the private key corresponding to the public key sent in the client’s certificate. However, as the client’s private key is intentionally private and not shared with other entities, it is not cryptographically possible to produce the required digital signature, thus breaking the mTLS client certificate authentication process at the middle device. There are, however, two scenarios where decrypting mTLS traffic can work:

  • The middle device has access to the client’s private key, or possesses an alternate public/private key pair for the user(s). For small environments this might be acceptable, but for anything more than a few clients this becomes extremely burdonsome to manage. Plus, if the clients are using smartcard-based certificate authentication, it would not be possible to extract the private key to pass to the middle device.

  • The middle device can “forge” a client certificate. This is where the middle device performs mTLS authentication with the client, then separately forges an identical version of the client’s certificate to present to the server, with a private key known to the middle device. On the F5 BIG-IP, this is a process called “Client Certificate Constrained Delegation” (C3D), and is described in more detail here: https://community.f5.com/t5/technical-articles/ssl-orchestrator-advanced-use-cases-client-certificate/ta-p/286005.

As previously noted, there are specific requirements for handling mTLS traffic passing through SSL Orchestrator in a reverse proxy:

  • To actively decrypt mTLS, implement Client Certificate Constrained Delegation (C3D) as described in the above article.

  • To bypass decryption of mTLS traffic, the SSL Settings of an SSL Orchestrator configuration contains the “Bypass on Client Cert Failure” setting. This allows the BIG-IP to detect a client certificate request from the server, and auto-initiate a TLS bypass for that traffic. This is a global setting and no other changes are required to bypass mTLS traffic through the reverse proxy.

  • To bypass decryption of mTLS traffic in a more granular way, the SSL Orchestrator security policy can define specific traffic matching criteria (ex. source or destination IP) to initiate the TLS bypass.