ltm rule command HTTP collect
iRule(1) BIG-IP TMSH Manual iRule(1)
HTTP::collect
Collects an amount of HTTP body data that you specify.
SYNOPSIS
HTTP::collect (CONTENT_LENGTH)?
DESCRIPTION
Collects an amount of HTTP body data, optionally specified with the argument. When the system collects the
specified amount of data, it calls the Tcl event HTTP_REQUEST_DATA or HTTP_RESPONSE_DATA. The collected data can be
accessed via the HTTP::payload command.
Note that this command cannot be called after any Tcl command that sends an HTTP response (e.g. redirect, HTTP::redirect,
and HTTP::respond). A run-time error will result.
Care must be taken when using HTTP::collect to not stall the connection. For example, some clients expect a response (such
as a "100 Continue" header) from the server before they will send data. To avoid a delay, you would need to send such a
response (possibly via TCP::respond) in your iRule. Also, if you use HTTP::collect without specifying a length, you must
have some non-HTTP event (e.g. AUTH_RESULT or NAME_RESOLVED) run HTTP::release, or HTTP processing will not continue, and
the collected data will be discarded when the connection times out.
It is important to note that these semantics are different than those of the TCP::collect and TCP::release commands. With
TCP::collect, the event for processing the data (CLIENT_DATA) will fire without TCP::release being called, whereas with
HTTP::collect, the event (HTTP_REQUEST_DATA or HTTP_RESPONSE_DATA) will not fire without HTTP::release being called. Again,
this is referring to using HTTP::collect without specifying a length. If you do specify a length, then the
HTTP_REQUEST_DATA or HTTP_RESPONSE_DATA event will fire without calling HTTP::release.
Note that although any size payload can theoretically be collected, the maximum size of a Tcl variable in v9 and v10 is 4MB
with a smaller functional maximum after charset expansion of approximately 1Mb. In v11, the maximum variable size was
increased to 32Mb. Any payload manipulation outside of calls to HTTP::payload should obey that limit.
The second example below includes the best practice logic to enforce that limit, including the suppression of response
chunking to allow more accurate determination of collect length. Even though it is possible to collect up to store up to
32Mb of data in a variable, it is not recommended to do this without careful consideration. If each connection can use 32Mb
of memory, TMM can quickly run out of memory.
Note that if multiple iRules invoke HTTP::collect simultaneously, (perhaps by being called by the same event in multiple
iRule scripts) then the result is undefined. This is because the amount of payload collected for the HTTP_REQUEST_DATA or
HTTP_RESPONSE_DATA event cannot satisfy the perhaps differing amounts wanted by the callers. iRules should arbitrate
amoungst themselves to prevent this situation from occuring, and have only one HTTP::collect call outstanding at a time.
Syntax
HTTP::collect
* Collects the entire HTTP body. Use caution when omitting the value
of the content length.
HTTP::collect
* Collects the amount of data that you specify with the
argument. Use caution when specifying a value larger than the size
of the actual length. Doing so can stall the connection. If the
specified amount of data exceeds the amount in the Content-Length
response header, only the smaller amount will be collected.
RETURN VALUE
VALID DURING
AUTH_ERROR, AUTH_FAILURE, AUTH_RESULT, AUTH_SUCCESS, AUTH_WANTCREDENTIAL, CACHE_REQUEST, CACHE_RESPONSE, HTTP_REQUEST,
HTTP_REQUEST_DATA, HTTP_REQUEST_SEND, HTTP_RESPONSE, HTTP_RESPONSE_DATA
EXAMPLES
when HTTP_RESPONSE {
if {[HTTP::status] == 205}{
HTTP::collect [HTTP::header Content-Length]
}
}
# Collect a request payload
when HTTP_REQUEST {
if {[HTTP::method] eq "POST"}{
# Trigger collection for up to 1MB of data
if {[HTTP::header "Content-Length"] ne "" && [HTTP::header "Content-Length"] <= 1048576}{
set content_length [HTTP::header "Content-Length"]
} else {
set content_length 1048576
}
# Check if $content_length is not set to 0
if { $content_length > 0} {
HTTP::collect $content_length
}
}
}
when HTTP_REQUEST_DATA {
# do stuff with the payload
set payload [HTTP::payload]
}
# Collect a response payload
when HTTP_REQUEST {
# Prevent the server from sending a compressed response
# remove the compression offerings from the client
HTTP::header remove "Accept-Encoding"
# Don't allow data to be chunked
if { [HTTP::version] eq "1.1" } {
# Force downgrade to HTTP 1.0, but still allow keep-alive connections.
# Since 1.1 is keep-alive by default, and 1.0 isn't,
# we need make sure the headers reflect the keep-alive status.
# Check if this is a keep alive connection
if { [HTTP::header is_keepalive] } {
# Replace the connection header value with "Keep-Alive"
HTTP::header replace "Connection" "Keep-Alive"
}
# Set server side request version to 1.0
# This forces the server to respond without chunking
HTTP::version "1.0"
}
}
when HTTP_RESPONSE {
# Trigger collection for up to 1MB of data
if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"]<= 1048576}{
set content_length [HTTP::header "Content-Length"]
} else {
set content_length 1048576
}
# Check if $content_length is not set to 0
if { content_length > 0} {
HTTP::collect $content_length
}
}
when HTTP_RESPONSE_DATA {
# do stuff with the payload
set payload [HTTP::payload]
}
HINTS
SEE ALSO
CHANGE LOG
@BIGIP-9.0.0 --First introduced the command.
BIG-IP 2022-04-12 iRule(1)