4.5. Managing ICAP Policies

4.5.1. What it is

Among all of the security service types, ICAP processing typically drives the most latency. SSL Orchestrator, as the ICAP client, must pass the request and response payload to an ICAP server and wait for a response. It is often useful, therefore, to limit what gets sent to the ICAP server and when it is sent. This is accomplished using the ICAP filters feature in SSL Orchestrator. An ICAP filter is an LTM policy object that enables and disables the ICAP Adapt profiles based on qualities of an HTTP request and/or response. The default action in an SSL Orchestrator ICAP service configuration is to send all traffic to ICAP. A more efficient approach to an ICAP filter is to disable ICAP when traffic does not match a required flow type.



4.5.2. How to build it

There are many different scenarios for disabling ICAP processing. This topic will only cover some of the more common use cases. There are also many different ways to accomplish each of the below use cases, so the following only covers one possible method for each use case. ICAP policies are defined in the BIG-IP UI under Local Traffic –> Policies. Click the Create button to create a Draft policy.

ICAP Policy Creation

User Input

Policy Name

Provide a unique name for the policy.

Strategy

The strategy determines how and when rules are evaluated. For best results, ICAP policies should normally use the first matching rule.

Type

Select Traffic Policy.


Click the Create Policy button, then click the Create button to add a new rule.


Scenario 1: Enabling ICAP only for HTTP response traffic

It may be useful in some scenarios to only send HTTP response traffic to ICAP, as the vast majority of HTTP GET requests are devoid of payloads.

ICAP Policies

User Input

Rule

Match all of the following conditions:

All Traffic

Do the following when the traffic is matched:

[Disable] [request adapt] at [request] time.


Click the Save button, then Save and Publish Policy. This rule simply disables the Request Adapt profile, leaving only the Response Adapt profile for handling response traffic.



Scenario 2: Enabling ICAP only for HTTP POST request traffic

POST requests will generally always contain a payload, while GET requests usually will not.

ICAP Policies

User Input

Rule 1

Match all of the following conditions:

[HTTP Method] full string [is] [any of] [POST] at [request] time.

Do the following when the traffic is matched:

[Enable] [request adapt] at [request] time.

[Disable] [response adapt] at [response] time.

Rule 2

Match all of the following conditions:

All Traffic

Do the following when the traffic is matched:

[Disable] [request adapt] at [request] time.

[Disable] [response adapt] at [response] time.


Click the Save button, and then Save and Publish Policy. Given the “first” strategy applied to the policy, rules are processed in ordinal order (ex. 1, 2, 3, etc.), and processing stops when a rule is matched. In this case, the first rule (1) looks for a POST request, then enables request adapt and disables response adapt. The second rule (2) then serves as a “catch all” when the first rule is not matched and disables request and response adapt.



Scenario 3: Enabling ICAP only for HTTP POST request and all response traffic

This rule will enable ICAP processing on POST requests, all HTTP responses, and would typically be used to filter out normally empty GET and other types of requests.

ICAP Policies

User Input

Rule 1

Match all of the following conditions:

[HTTP Method] full string [is] [any of] [POST] at [request] time.

Do the following when the traffic is matched:

[Enable] [request adapt] at [request] time.

Rule 2

Match all of the following conditions:

All Traffic

Do the following when the traffic is matched:

[Disable] [request adapt] at [request] time.


Click the Save button, and then Save and Publish Policy. Given the “first” strategy applied to the policy, rules are processed in ordinal order (ex. 1, 2, 3, etc.), and processing stops when a rule is matched. In this case, the first rule (1) only matches on a POST request. As policy processing stops on this matching rule, response adapt is still active, so the rule enables request adapt for POST and all HTTP response traffic. The second rule (2) serves as a “catch all” when the first rule is not matched and only disables HTTP request traffic processing.



Scenario 4: Enabling ICAP for specific request URL extensions and response content types

Sometimes you just need to activate ICAP for specific file type requests - request URLs that end with something like “.exe”, “.sh”, or “.bat”, and also only scan responses of a specific MIME types (HTTP Content-Type) like “application/json” and application/executable . This can be accomplished by creating data groups to list out the different URL endings and MIME types. Data groups are created under Local Traffic –> iRules –> Data Group List.

Document Type Data Group

User Input

Name

Provide a unique name (ex. /Common/doc-type).

Type

Select String.

String Records

Only the String entries are required. The corresponding Value entry is not required and can be used as a description field or simply left blank. For example, if you want to inspect specific document types presented in URL requests, add a few of the following:

.exe

.sh

.bat

.tar

.tgz

.zip


Mime Type Data Group

User Input

Name

Provide a unique name (ex. /Common/mime-type).

Type

Select String.

String Records

Only the String entries are required here. The corresponding Value entry is not required so can be used as a description field or simply left blank. For example, if you want to inspect responses with specific Content-Type values in the HTTP response, add a few of these:

application/json

application/gzip

application/zip

application/x-tar

For a broader list of common MIME types, see: https://developer.mozilla.org/en-US/docs/Web /HTTP/Basics_of_HTTP/MIME_types/Common_types


Now create a policy to use these data groups.

ICAP Policy Creation

User Input

Policy Name

Provide a unique name for the policy.

Strategy

The strategy determines how and when rules are evaluated. Select the all matching rule.

Type

Select Traffic Policy.


ICAP Policies

User Input

Rule 1

Match all of the following conditions:

[HTTP Method] full string [exists] at [request] time.

Do the following when the traffic is matched:

[Disable] [request adapt] at [request] time.

Rule 2

Match all of the following conditions:

[HTTP URI] [full string] [contains] [in datagroup] [/Common/doc-type] at [request] time.

Do the following when the traffic is matched:

[Enable] [request adapt] at [request] time.

Rule 3

Match all of the following conditions:

[HTTP Status] [full string] [exists] at [response] time.

Do the following when the traffic is matched:

[Disable] [response adapt] at [response] time.

Rule 4

Match all of the following conditions:

[HTTP Header] full string named [Content-Type] is [in datagroup] [/Common/mime-type] at [response] time.

Do the following when the traffic is matched:

[Enable] [response adapt] at [response] time.


This one definitely deserves some deeper analysis. Notice first that the strategy for the policy was switched to all. As the requirement is to match something OR something else, this strategy is more suited:

All: Specifies that the system starts the actions belonging to all rules in the Rules list that match. If a conflict occurs, for example, one rule matches to pool A and another rule matches to pool B. The system uses the best matching rule strategy to resolve the conflict. A best-match rule can be the lowest ordinal, the highest priority, or the first rule that matches in the rules list.

Best: Specifies that rules are not ordered. Instead, the system selects and starts the actions of the rule in the Rules list with the best match. When multiple rules match and specify an action, conflicting or otherwise, only the action of the best matching rule is implemented. A best matching rule can be the lowest ordinal (determined by the precedence of the operand), the highest priority, or the first rule that matches in the Rules list.

So basically, all rules are processed, and if there is a conflict, the better rule is implemented. To better understand how this works, here a few examples. Assume the above data groups exist with the described values.

  • https://www.example.com/

    This is an HTTP GET request to www.example.com/, which will return a response of type “text/html”. In this case, the following rules are processed:

    • Rule 1 - disable request adapt

    • Rule 3 - disable response adapt

    So, for this request, all ICAP processing is disabled.


  • https://www.example.com/?foo.exe

    This is an HTTP GET request to www.example.com/, which will return a response of type “text/html”. The request URL also ends with “.exe”, which matches the doc-type data group. In this case, the following rules are processed:

    • Rule 1 - disable request adapt

    • Rule 2 - enable request adapt because the URL matches the data group

    • Rule 3 - disable response adapt

    Rules 1 and 2 both match but because of their ordinal order, rule 2 wins so request adapt is enabled for this request.


  • https://httpbin.org/headers

    This is an HTTP GET request to httpbin.org/headers, which will return a response of type “application/json”. This response mime type matches the mime-type data group, so in this case the following rules are processed:

    • Rule 1 - disable request adapt

    • Rule 3 - disable response adapt

    • Rule 4 - enable response adapt because the MIME type matches the data group

    Rules 3 and 4 both match but because of their ordinal order, rule 4 wins so response adapt is enabled for this response.


  • https://httpbin.org/headers/?foo.exe

    This is an HTTP GET request to httpbin.org/headers/?foo.exe, which returns a response of type “application/json”. The response mime type matches the MIME type data group, but also, the request URL matches the doc-type data group. In this case, the following rules are processed:

    • Rule 1 - disable request adapt

    • Rule 2 - enable request adapt because the URL matches the data group

    • Rule 3 - disable response adapt

    • Rule 4 - enable response adapt because the mime type matches the data group

    Rules 1 and 2 both match but 2 wins because of its ordinal position, so request adapt is enabled. Rules 3 and 4 match, but 4 wins because of its ordinal position, so response adapt is enabled.

Note that the rules that disable adapt both use an “exists” condition, as opposed to simply applying to all traffic. This is also required to ensure that the “better” rule wins. Suffice it to say, there are still many more use cases for ICAP policy filtering, and enumerable ways to achieve these use cases. Hopefully the above examples provide solid ground to start building effective ICAP policies for your SSL Orchestrator topologies.


4.5.3. Additional Information

LTM CPM policies for ICAP filtering are powerful, but as with any UI-driven configuration system, may still have some limitations. You can extend ICAP filtering logic far beyond the UI’s capabilities by employing an alternate iRule solution. To use iRules for ICAP filtering, disable strictness on the ICAP service and add an iRule to the service’s corresponding “-t-4” LTM virtual server. The objective of the iRule is to enable or disable the ADAPT profiles (as CPM does above). The below is an example iRule for effective ICAP traffic filtering.

Note that with this iRule option, the ICAP service must remain in non-strictness. Also note, as with LTM CPM ICAP policies in an HA configuration, you must manually sync the BIG-IP configuration between the SSL Orchestrator peers before making changes to the SSL Orchestrator configuration.

when HTTP_REQUEST {
   ## Set default DLP state (0=disabled)
   set dlp_cmd 0

   ## Only do ICAP filtering for POST and PUT requests
   if { [lsearch [list "PUT" "POST"] [HTTP::method]] >= 0 } {
      set dlp_cmd 1

      ## And don't enable ICAP if the Content-Length header is over 1 MB
      if { ( [HTTP::header exists Content-Length] ) and ( [expr { [HTTP::header Content-Length] >= 1100000}] ) } {
            set dlp_cmd 0
      }

      ## Test for other things (IP, domain name, URL path)
      ## Use 'set dlp 0' to disable ICAP
      ## Use 'set dlp 1' to enable ICAP
   }

   ## Enable or disable ADAPT (ICAP) based on above evaluations
   if { ${dlp_cmd} } {
      ADAPT::enable request true
      ADAPT::enable response true
   } else {
      ADAPT::enable request false
      ADAPT::enable response false
   }
}