Project BAIU - Basic to Advanced IP & UserID Rate Limiter¶
Here is Project BAIU! The Basic(Bad) to Advanced(Ass) IP & UserID Rate
Limiter! This is a world first rate limiter that thus far (August
2014) can
not be matched.
Currently we have our patent pending in process for this rate limiter
but
it is royalty free to use for anyone. The patent will be licensed to
Company ABC International, Inc. Below I will first include the patent
filing paperwork which
explains in depth what this is rule set is for and why it is so
awesome!
Attached will be the iRules themselves and the sync scripts I use to
sync
this across multiple boxes.
This isn’t just an iRule, it is a suite of iRules and shell scripts
that
complete what could be seen as an entire product. I received the F5
DevRockStar award this year for this iRule suite as well and F5 has
even
considered it for their production appliance so soon enough you might
just
be using this, but here you can control it anyway you want. It should
work
for any website as well with a few small configuration changes. You
also
don’t need anything special licensed, just the F5 and one of the
products
that allows iRules. The UID extraction should be universal as well at
worst
requiring one parameter adjustment which is doubtful.
In one year we have reduced our point based theft from over $8,000 a
week
average, to $0 which was a total savings of nearly $.5 million in
revenue.
The accuracy still remains at over 99.99% accurate meaning less than
.001%
false positives.
All iRules and scripts are attached except the basic blacklist iRule
which
will not attach due to F5’s security policy for DevCentral. Not sure
why
but I included that in the zip file which contains everything in one
download.
There are several iRules that are included and everything is focused
around
the client IP value being an XFF header value from Akamai which is
called
“True-Client-IP”. I also include the logic that pulls that header and
converts the header into the client IP used for the rest of the
ASM/iRules
rather than say the load balancer/proxy IP forwarding the traffic to
the
F5. The iRules all start with the name XXX_ and are to be loaded in
that
order. Priorities inside the iRules override the order even if you
mess
that up. iRules: XXX_010_TIP.txt
XXX_030_Blacklist_IP_UserAgent_Referer.txt
XXX_040_UIDExtractAllSites_AddHeader.txt XXX_060_BAIU_Tru_R8L.txt
XXX_071_BAIU_CnC.txt
There also also several shell scripts. These scripts can be loaded as
cron
jobs and will sync the ban and request counts across multiple boxes
whether
you have 1 or 2000, it doesn’t matter. To add a sync box in, just
change
the IP’s accordingly to where you have the CnC iRule installed. The
scripts
use NetCat to perform the sync which is a binary included on all F5
appliances meaning you will remain support using these cron jobs as
well. The cron jobs are named for how often they should run otherwise
they
should be hourly to daily depending on user’s desire.
Scripts: rollingban_sync.txt banlist_sync.txt reqcnt_hourly_sync.txt
reqcnt_10min_sync.txt
Enjoy!
(a) Problem To Be Solved By The Invention / Purpose Of The Invention:
The problem being solved is the inability of web sites to
differentiate
criminals from legitimate users. The specific example that led to this
invention is the use case of criminals taking advantage of the
inability to
discern legitimate from malicious users and hacking into user
accounts,
gaining access to sensitive user data, and stealing credits or points
that
can, through various methods, be converted to money.
Criminals are using a highly diverse set of computers in their
attacks.
These computers are likely normal computers that have been compromised
with
malicious software. This malicious software, also known as “malware”,
is
often running unknown to the owner of the computer.
The criminals also have lists of hacked user names and passwords that
people have set up for various website accounts, such as social
networking
and travel sites. This gives the criminals a vast, varied, and
seemingly
legitimate set of computers on the internet, and also lists of
usernames
and passwords to one or more websites. With this, the criminals then
launch attacks from this network of computers, against various
websites,
using those or similar usernames and passwords. Since many people use
the
same or similar usernames and passwords across multiple accounts, the
criminals are often able to get access to user accounts on these
various
websites.
Identifying and blocking these criminals remains a current security
challenge for companies that provide web content and have something of
value that is protected by means of user authentication. This
invention
can potentially be used by anyone running a website that needs to
differentiate between legitimate and malicious users.
(b) How the problem is currently solved: Currently, Transaction Rate
Limiters (TRLs) and Brute Force Prevention
Utilities (BFUs) have limited user recognition functionality which
relies
on correlating and identifying traffic trends based on IP address and,
in
some cases, a userID/string element, for a static, per moment, event;
“here
and now”. The TRLs and BFUs monitor events in transactions per second,
per
minute, and/or per hour. The attacking group(s) are skilled enough to
work
around this limited functionality and have an attack footprint that
utilizes thousands of source IP addresses and span time periods of
hours,
days, and weeks.
Example: 3,000 IPs with a single TRL with a limit of 10 transactions
per second
allows for 2,592,000,000 chances for a successful attack.
(c) How Invention Solves The Problem:
The BAIU (Basic to Advanced IP & UserID) Rate Limiter is designed to
actively monitor, and/or restrict the ability of network based
entities,
both human and automated software, accessing certain aspects of a
hosted
site that is viewable on the internet over periods of hours, to days,
to
weeks, to months with multiple, user-defined, variable control methods
for
restriction, event triggers, multiple methods of counting IP and/or
UserID
access, and multiple variable, user-defined timing controls that set
independent timing constraints for monitoring and restricting
IP/UserID
access. It is capable of monitoring and restricting via a static and
dynamically distributed model when monitoring and restricting IPs
and/or
userID/strings.
Monitoring and restricting over customizable periods of time (hours,
days,
weeks, months) significantly reduces the daily chances of attacks.
Rather
than the “here and now” approach commonly used, BAIU registers,
catalogs,
analyzes, trends, and consider behavioral characteristics to determine
the
appropriate reaction. This is the core of the static monitoring and
restriction functionality.
Begin Section C Reference (1): With the example defined in response
(B) and BAIU configured with a limit
of 12 userIDs per IP, and 6 IPs per userID, 40 requests per IP, 30
requests
per userID, and one instance of BAIU we now have a maximum daily
chance at
attacking 36,000 unique accounts, with a maximum possibility of
chances of
successful attack being 120,000.
This is over a 99.99% reduction of chances from a non-BAIU protected
system. These are the highest mark numbers possible, however due to
the
behavioral characteristics BAIU is capable of correlating, the number
of
chances are significantly reduced even further. The key concept to
BAIU is
the harder an attacker typically works to compromise the website, the
better and faster BAIU works against them.
For example, if 1 userID is called by 4 different IPs one time, all 4
IPs
and the userID are now blocked. This eliminates 5 threat vectors
reducing
the maximum chances of unique userIDs by 66 in 5 requests. Depending
on
configuration, this also now means all subsequent IPs trying the
blocked
userID could now be blocked. If 12 IPs call the blocked UID, the
maximum
chances of unique userIDs attempts are now reduced 144.
When adding theory of probability the maximum chances are even further
reduced when using the behavioral/anomaly capabilities of BAIU raising
the
possibility to reduce threats to a theoretical 100% over time which is
un-
claimable by any other system.
In most attack scenarios, whether distributed, dynamic and
distributed, or
just brute force, BAIU handles the scenarios effectively and according
to
configuration with exceptional accuracy and results.
Currently with 23,000~ IPs and about 36,000~ unique userIDs monitored
daily, BAIU (two instances across two F5 3900 appliances actively
syncing
their ban/request list every 5 minutes and 59th minute of the hour)
actively ban roughly 80~ unique IP and about 70~ unique userIDs daily
for
all production Company ABC International, Inc.’s consumer web sites.
End Section C Reference (1).
(d) Specific Features that Make this Invention Novel:
UserID/string simplification to ensure tracking of only useful
IDs/strings,
and ensure variances of the same element is tracked as one rather than
uniquely.
Traditional TRLs and BFUs have a static approach when tracking a
userID/string element to monitor. To support accurate anomaly
detection,
BAIU uses a simplification process when monitoring a string to ensure
almost any combination of a string value that would still be
associated
with a single account element, will be handled as a single element
rather
than multiple variations that could be handled uniquely. String
manipulation/simplification functionality is all Configuring
User/Organization (CU/O) defined. The inventor’s standard approach
would
be to set all characters to lower case, remove the @ character and all
trailing string elements leaving only the user ID portion, then remove
all
special characters and punctuation, setting all digits to 0 or like
character, finally having a simplified ID to now track. This is just
one
approach, and other concepts or variances of this setup can be used.
The simplification process is to ensure a single user who has mistyped
their userID/string element or is trying multiple variations of what
would
likely be perceived as the same userID/string element remains as a
single
element. It also negates chances of tracking userIDs/strings that do
not
conform to a site’s userID/string standard such as too long, too
short, and
even de-obfuscating code based attack strings such as a SQL Injection
attack. This ensures the least, but most effective elements, are
tracked
and available for monitoring and/or restriction. The manipulation is
only
done for the monitoring and restricting device as well so that the
tracking
string is not manipulated at the TCP/UDP packet level, only in device
memory.
An example of userID simplification would be as follows:
“BiLBo_._HoPPiNS”, “B.I#.!lB(O)+=HoP!$,.P?i/nS”, will all be tracked
as
“bilbohoppins”.
This is part of the dynamic monitoring and restriction functionality.
Track/correlate multiple IPs querying a single userID/string, and
track/correlate a single IP querying multiple UserIDs/strings. This
ensures
events that are relevant/similar through correlation, will be handled
as
one, accordingly (Many to One).
Traditional TRLs and BFUs typically consider events either per IP, or
per
userID/string. BAIU monitors and restricts the limit of userID/strings
called by a single IP. It also monitors and restricts the number of
IPs
that have called a userID/string. This ensures a dynamically
distributed
chain of events for an IP/userID/string is more likely to be treated
as a
singular set of entities and events, rather than separate entities and
events. A potential standard would be a limit of 15 userIDs per IP,
and 6
IPs per userID. Anything equal to, or more, would invoke a restriction
event. This is the core of the distributed dynamic monitoring and
restriction functionality.
IP Geo-location service inclusion for IP locality correlation (Many to
One). BAIU is capable of utilizing IP Geo-location logic in the Many
to One
correlation described above. For example, BAIU can perform zip code
analysis and, if zip codes differ by a defined amount, invoke a
monitoring
or restricting event.
Basic to advanced defined logic to detect static or dynamically
distributed
events that can be configured by a CU/O to meet the specific trends
and
criteria that would deem traffic questionable/malicious/undesired.
Event triggers are defined uniquely and as extensively as the CU/O
chooses.
Event triggers could be any defined HTTP/TCP/UDP based event(s) that
would
be deemed as malicious/negative/undesired by the CU/O. Examples could
be
an HTTP response code, or the existence/non-existence of a cookie.
Malicious/negative/undesired events will cause the request counter for
the
identified IP/userID/string element to increase by 1 and by the amount
set
in the multiplier value. The purpose is to have valid sessions have a
standard request rate of 1, while other events that meet a
malicious/negative/undesired event’s criteria have a standard request
rate
of greater than 1. This ensures malicious/negative/undesired elements
reach a given threshold much faster than normal. This is part of the
dynamic monitoring and restriction functionality.
Customizable time-periods for monitoring and restricting based on
IP/UserID/String and their restricting or monitoring events.
Restrictions are fully definable for IP and userID/string values with
features that can be enabled and disabled during run time. The
standard
approach is for IPs and userID/strings to have independent timing
values
that can be chosen from three timing value sets; Standard, Half, and
Long.
Each timing value set has timing values for banning and monitoring.
More
or less timing values can be used depending on need of complexity.
The standard approach to restriction configuration is to restrict for
a
Standard ban time and add the given element to the Long period of time
tracking. Upon release of restriction, if the element continues to hit
the
thresholds configured to invoke restriction within the Long period of
monitoring, then the Long ban value will be used and the Long timer
will be
reset. Single occurrence offenders are released in a standard time,
while
repeat offenders will be restricted for the Long time period.
Specific triggers to increase counters more rapidly to ensure an
element is
moved to a restricting event sooner upon CU/O defined criteria that is
deemed malicious. A multiplier is used for event triggers to define
the value that a request
should be incremented by to effectively increase the request counts
faster
for the actual request. Event action has a CU/O setting to trigger
quicker
action. This is the core to the anomaly detection.
Configurable monitoring and restriction thresholds. Restriction and
event actions can be initiated based on configured
thresholds per IP and UserID/string. Thresholds are also considered
when
setting multiplier values.
Specific events for post-restricting activity, such as resetting
counters
for repeat traffic during restriction. A further expansion on
restriction capabilities is the ability to generate
a new action upon events that occur during the restriction time frame.
For
example, restriction time can be based on the most recent event
ensuring a
“rolling” restriction occurs and the element is never dropped from
restriction until that element no longer generates events during the
configured time period. Additionally, there is an ability to restrict
all
IP/userID/string elements that call an element that has already been
restricted. Other events besides a restriction event can be defined
instead, with restriction being the standard action.
Manual interaction/manipulation of monitored and restricted elements
by the
CU/O. A set of Administration URIs that are custom defined by the CU/O
allow
administration of the monitoring and restriction settings for
IP/userID/string elements. These URIs give specific control to add,
remove, and edit entries that are actively being monitored and
restricted.
This gives granular manual control to the CU/O beyond the automated
capabilities. The primary function of the Administration URI’s is to
add and remove
elements from the monitoring and restriction tables. However, CU/O can
define capabilities such as setting request counts to maximum to
ensure
given element makes more interaction with the BAIU rate limiter which
allows other counters the chance to increase if said elements invoke
any
events.
An example would be to max an IP’s request count so that when it makes
another connection, the userID element is also considered for
monitoring
and/or restriction.
Synchronization of monitoring and restricting events across multiple
platforms that the mechanism has been built/cloned on to create
distributed
load while ensuring events are tracked equally. The Administration
URIs can be directly accessed within the site’s defined
control parameters and called via each device (BAIU capable or not) at
a
manual or timed interval through such means as a cronjob. The
devices(s)
parse through all logged restriction events within a CU/O defined
interval,
and at a said interval defined by the CU/O, will call all other BAIU
functional devices via the Administrative URI for that function to
add/drop
all restricted/dropped IP/userID/string elements. This effectively
allows
synchronization of all restricted/dropped elements across all devices
that
have BAIU enabled.
Real-time monitoring of all events can be configured as well to ensure
real-time synchronization using the same methods or a combination
there-of
could be used depending on the CU/O’s needs.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Use Case Information: Comparative analysis does not exist for the
capabilities and elements
available for monitoring over the length of times specified. So far an
F5
3900 appliance (Specifications available at
www.f5.com/pdf/products/big-
ip-platforms-datasheet.pdf) has been used and it has been proven
successfully capable of tracking 116,280 unique elements over 7 days,
and 8
hours successfully, accurately, at an atomic clock level (thanks to F5
appliance) without any increase in standard CPU or RAM/memory usage
upon
memory usage/consumption that occurs normally by design via the Linux
core
kernel memory usage standards. The appliances have never broken 68.2%
memory usage with an average reserve load of 66.6% which is standard
with/out BAIU being loaded i.e. standard production usage. 2.8k~
connections per second have been successfully loaded through BAIU
over a week time period on several test scenarios with no performance
impact experienced as well with the F5 3900 appliance. The appliances
running BAIU remain fully functional without any issue after
4+ months running. Please refer to Section C Reference (1) for
additional commentary.
(e) Other Inventions That Address The Same Problem: F5 ASM provides
“Brute Force Prevention” functionality in an attempt to
address the same problem as this invention. There are also other
various
rate limiters and brute force remediation tools that offer limited
analysis, correlation, and response functionality.
(f) Shortcomings/Disadvantages Of Other Inventions: Other products,
such as the F5 ASM, do not offer all of the malicious
traffic identification logic, the static and dynamic control methods,
the
userID simplification functionality, or the granular configuration and
user
control options and flexibility. When configuring this feature with a
similar set of thresholds as defined
in Section C Reference (1), it was incapable of restricting a single
event
BAIU restricted in production traffic due to the method of throttling
used
by the attacking parties.
==The Actual Code:==
iRules:¶
==XXX_010_TIP.txt==
# This iRule is a universal rule that applies to all configurations.
# It will pull the true client IP value from the X Forwarded For header
values.
# and then inject it into the headers.
# This iRule should be placed first before all other iRules to ensure the
proper IP is
# used when taking action. This iRule also ensures the localhost IP will
not be blocked.
#
# Dependencies: NONE iRules / NONE DataGroups
when CLIENT_ACCEPTED priority 1 {
# Null out all universal variables
set tip ""
set allowed ""
set waf_uid ""
set sid ""
set jsid ""
set uri ""
set r8luri ""
set loginuri ""
set lp1 ""
set lp2 ""
set lp3 ""
set lp4 ""
# 0 - NONE, 1 - LOW, 2 - MEDIUM, 3 - VERBOSE
set DEBUG_LEVEL 1
}
when HTTP_REQUEST priority 10 {
if {[HTTP::header "True-Client-IP"] ne ""} {
# The last True-Client-IP header will be grabbed
set xff [split [string map [list " " ""] [HTTP::header "True-
Client-IP"]] ","]
# If the header ends with 127.0.0.1 but is in a list of IPs then
set the TIP as the value before 127.0.0.1
if {$xff ends_with " 127.0.0.1"} {
set tip [lindex $xff end-1]
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: IP: $tip -
127.0.0.1 re-written" }
} else {
# If TIP appears normal then write TIP value as last IP value
set tip [lindex $xff end]
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: IP: $tip - TIP
value written" }
}
# If TIP equals 127.0.0.1 then re-write to avoid blocking
WAF TMM processes
if {$tip eq "127.0.0.1"} { set tip "169.254.254.200" }
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: IP: $tip -
Saved 127.0.0.1 re-wrote TIP " }
HTTP::header remove "True-Client-IP"
HTTP::header insert True-Client-IP $tip
} else {
set tip "169.254.254.254"
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: IP: $tip - NO Tip 169
TIP written" }
HTTP::header insert True-Client-IP $tip
}
set allowed 1
if {$allowed ne "1"} {
# Block this request
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: Blocking Request for
$tip - [IP::client_addr]" }
# Change the below line to "drop" to drop the TCP connection
instead
HTTP::respond 403 content {<html>blocked</html>} noserver
# Close TCP connection so client can't make further requests
reject
} else {
#Request has been allowed
if { $DEBUG_LEVEL >= 3 } { log local0. "TIP: IP: $tip - " }
}
}
==XXX_030_Blacklist_IP_UserAgent_Referer.txt==
#Blacklist IPs User Agents and Referrers
#
when CLIENT_ACCEPTED {
# Check to make sure that the following classes exist before using
this iRule:
# -Note that the below data groups should have their data
stored in the data group key name. The datagroup
# key value is not used
# blacklist_IPs - Disallowed true client IP addresses
# blacklist_user_agents - Disallowed user agent headers - all
entries should be lowercase
# blacklist_referer - Disallowed referer headers - all entries
should be lowercase
# NOTE - Turn off logging with debug set to 0. Turn on with 1
# 0 - NONE, 1 - LOW, 2 - MEDIUM, 3 - VERBOSE
set DEBUG_LEVEL 1
set n_tip ""
set n_user_agent ""
# #set n_refer ""
set n_sid ""
set allowed ""
}
when HTTP_REQUEST {
set t_user_agent [string tolower [HTTP::header "User-Agent"]]
# #set t_refer [string tolower [HTTP::header "Referer"]]
# Set user cookie as Session ID
set sid [string range [HTTP::cookie sid] 0 18 ]
set jsid [string range [HTTP::cookie JSESSIONID] 52 75 ]
# Set URI value
set uri [HTTP::path]
set allowed 1
# The below if blocks are for blacklisting functionality
# Some of the if branches below are redundant, but this setup
allows for more granular logging
if { [class match $tip equals blacklist_IPs] } {
set n_tip $tip
set allowed 0
if { $DEBUG_LEVEL >= 3 } { log local0.
"BLACKLISTING: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~
Blacklisted client IP matched: $n_tip" }
}
if { [class match $t_user_agent contains blacklist_user_agents] } {
set n_user_agent $t_user_agent
set allowed 0
if { $DEBUG_LEVEL >= 3 } { log local0. "BLACKLISTING: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Blacklisted user agent
matched: $n_user_agent" }
}
if { [class match $sid equals blacklist_sid] } {
set n_sid $sid
set allowed 0
if { $DEBUG_LEVEL >= 1 } { log local0. "BLACKLISTING: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Blacklisted SID matched:
$n_sid" }
}
if { [class match $jsid equals blacklist_sid] } {
set n_sid $jsid
set allowed 0
if { $DEBUG_LEVEL >= 1 } { log local0. "BLACKLISTING: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Blacklisted JSESSIONID
matched: $n_sid" }
}
# #if { [class match $t_refer equals blacklist_referer] } {
# #set n_refer $t_refer
# #set allowed 0
# #if { $DEBUG_LEVEL >= 3 } { log local0.
"BLACKLISTING: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~
Blacklisted referer matched: $n_refer" }
# #}
if {$allowed ne "1"}{
# Change the below line to "drop" to drop the TCP
connection instead
HTTP::respond 403 content {<html> ?(?????)? blocked ¯\_(?)
_/¯ </html>} noserver
# Close TCP connection so client can't make further
requests
reject
} else {
#Request has been allowed
if { $DEBUG_LEVEL >= 3 } { log local0. "ALLOWING-
BLOCKING: Request allowed for $tip" }
}
}
XXX_040_UIDExtractAllSites_AddHeader.txt
# This extracts user IDs from a login URI. It's likely this will extract
from most websites but could require some adjustments
# Most likely to the amount of content pulled. The UID attempts twice for
two different styles of UID parameters
# and then logs a failure and inserts a UID-NULL value for a fail safe.
# Dependencies: TIP_Univ iRule / XXX_LOGIN_URI R8L_BAD_IP DataGroup
# !!!R8L_GOOD_UID must have UID-NULL added to avoid banning unextractable
usernames!!!
when HTTP_REQUEST priority 40 {
# Set Login Pages
set login_page XXX_LOGIN_URI
# HTTP protocol methods that is used by the login form
set method [HTTP::method]
# 0 - NONE, 1 - LOW, 2 - MEDIUM, 3 - VERBOSE
set DEBUG_LEVEL 1
# Set user cookie as SID/JSESSIONID
set sid [string range [HTTP::cookie sid] 0 18 ]
set jsid [string range [HTTP::cookie JSESSIONID] 52 75 ]
# Only process matched uris & allow specific IPs/UIDs
if { ( [class match [HTTP::header "True-Client-IP"] equals
R8L_GOOD_IP ] ) } {
if { $DEBUG_LEVEL >= 3 } {log local0. "NO_MATCH: IP: $tip
SID: $sid~$jsid ~ IP Whitelisted"}
return
}
if { ( not [class match [HTTP::uri] equals XXX_LOGIN_URI ] ) and (
not [class match $tip equals R8L_BAD_IP ] ) } {
if { $DEBUG_LEVEL >= 3 } { log local0. "RETURN: IP: $tip
SESSSION: $sid~$jsid Returning from irule---no match on URI or IP found" }
return
}
HTTP::collect 450
# Log the debug message if collect is enabled
if { $DEBUG_LEVEL >= 3 }{log local0. "$tip Collecting HTTP
payload"}
}
# Got the data, parse them and generate the syslog message
when HTTP_REQUEST_DATA priority 41 {
# Pull username if present
if {$method == "POST" } {
# Log the debug message if trace is enabled
if { $DEBUG_LEVEL >= 3 }{log local0. "INSPECTPAYLOAD:
Client= $tip Collected request data: [HTTP::payload]"}
# Extract User ID from payload (1st attempt)
set waf_uid [ string trim [ string range [ findstr [string map
{\x2D\x2D\x2D\x2D\x2D \x26} [string map {\x22 \x26} [string trim
[HTTP::payload] ] ] ] username 9 & ] 0 20 ] ]
# Insert UID Header
HTTP::header insert "lws" UID-Extract $waf_uid
# Log UID success event
if { $DEBUG_LEVEL >= 1 } { log local0. "UIDSUCCESS: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: [HTTP::uri] ~ UID=$waf_uid" }
}
# Username not found with normal username parameter try other
if { $waf_uid equals "" } {
# Pull username if present
if {$method == "POST" } {
# Log the debug message if trace is enabled
if { $DEBUG_LEVEL >= 3 }{log local0.
"INSPECTPAYLOAD: Client= $tip Collected request data: [HTTP::payload]"}
# Extract User ID from payload (2nd attempt)
set waf_uid [ string trim [ string range [ findstr
[string map {\x2D\x2D\x2D\x2D\x2D \x26} [string map {\x22 \x26} [string
trim [HTTP::payload] ] ] ] profileName 12 & ] 0 20 ] ]
# Insert UID Header
# HTTP::header insert "lws" UID-Extract $waf_uid
# Log UID success event
if { $DEBUG_LEVEL >= 1 } { log local0. "UIDSUCCESS:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: [HTTP::uri] ~ UID=$waf_uid" }
}
# If User ID still not found then write UID-NULL
if { $waf_uid equals "" } {
set waf_uid UID-NULL
HTTP::header insert "lws" UID-Extract UID-NULL
if { $DEBUG_LEVEL >= 2 } { log local0. "UIDFAIL: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: [HTTP::uri] ~ UID=$waf_uid" }
}
}
}
==XXX_060_BAIU_Tru_R8L.txt==
# This iRule limits IPs & UIDs extracted from a session against URIs which
are configured separately in iRule datagroups
# where the names are respective to their function. Rate limits are done
within a set of values and then block by IP or UID after
# set thresholds. There are three levels of ban timers, StdB, LongB, &
HalfB though more can be added.
# There are three levels of timers, HTime STime & LTime. These cover
different timing lengths.
# There are IPMaxReqs, UIDMaxReqs, & URIMaxReqs which all set max request
values per type.
# I2UCnt and U2ICnt both have their own timers in the Hydra section.
# Dependencies: TIP_ALL, UIDExtract_AddHeader_ALL iRules / XXX_LOGIN_URI,
R8L_URI, R8L_BAD_IP, R8L_BAD_UID, R8L_GOOD_IP, R8L_GOOD_UID, XXX_LAN
DataGroups
when HTTP_REQUEST priority 50 {
###########################################################################
# Set user cookie as Session ID
set sid [string range [HTTP::cookie sid] 0 18 ]
set jsid [string range [HTTP::cookie JSESSIONID] 52 75 ]
# Set URI value
set uri [HTTP::path]
# 0 - NONE, 1 - LOW, 2 - MEDIUM, 3 - VERBOSE
set DEBUG_LEVEL 1
###########################################################################
# Only process matched uris & allow specific IPs/UIDs
if { ( [class match $tip equals R8L_GOOD_IP ] ) } {
if { $DEBUG_LEVEL >= 2 } {log local0. "WHITELIST_IP: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ IP Whitelisted"}
return
}
# # This line is enabled down below in the UID section. Disabling
here ensures an IP gets a Request Count.
# #if { ( [class match $waf_uid equals R8L_GOOD_UID ] ) } {
# #if { $DEBUG_LEVEL >= 2 } {log local0. "WHITELIST_UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Username Whitelisted"}
# #return
# #}
if { ( not [class match $uri equals R8L_URI ] ) and ( not [class
match $uri equals XXX_LOGIN_URI ] ) } {
if { $DEBUG_LEVEL >= 3 } {log local0. "NOMATCH_URI: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ URI not limited"}
return
}
if { [class match $tip equals R8L_BAD_IP ] } {
if { $DEBUG_LEVEL >= 2 } {log local0. "MATCH_BAD_IP: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ IP in BAD group"}
}
if { [class match $tip equals R8L_BAD_UID ] } {
if { $DEBUG_LEVEL >= 2 } {log local0. "MATCH_BAD_UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ UID in BAD group"}
}
###########################################################################
# /// Sword - Single IP/UID/URI Tracking/Banning - Cut Them Down!
###########################################################################
# Used to set different time outs for IP, UID, & URI tracking
# Ban and request times are for all but can make unique if desired
# Total IP Requests allowed
set IPMaxReqs 40
# Total UID Requests allowed
set UIDMaxReqs 30
# Total URI Requests allowed
set URIMaxReqs 1000
# Set Rate Limit Failed Auth Multiplier
set R8X 1
# Used for standard banning
# Requests within X seconds
set STime 115200
# Ban X seconds
set StdB 194400
# Used for short banning
# Requests within X seconds
set HTime 108000
# Ban X seconds
set HalfB 57600
# Used for longer duration banning
# Banned events within X seconds
set LTime 691200
# Total banned events
set MaxBs 0
# Ban X seconds
set LongB 453600
# # Build monitor table for tracking elements
# #set MONITOR_TBL uid_limit_monitor
# #set MONITOR_TBL ip_limit_monitor
# #set MONITOR_TBL uri_limit_monitor
###########################################################################
# This builds the Ban IP tracking value
set b_ip "b_$tip"
###########################################################################
# Distinctions
# This builds the R8L URI tracking value
if { [class match $uri equals R8L_URI ] } {
set r8luri 1
if { $DEBUG_LEVEL >= 2 } {log local0.
"R8L_URI_CALLED: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ R8L URI
called "}
}
# This builds the Login URI tracking value
if { [class match $uri equals XXX_LOGIN_URI ] } {
set loginuri 1
if { $DEBUG_LEVEL >= 2 } {log local0.
"LOGIN_URI_CALLED: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ LOGIN
URI called "}
}
if { $r8luri == 1 } {
###########################################################################
# Set Normal R8L URI Counters
set URICnt [table keys -subtable "URIReqMon" -count]
###########################################################################
# Begin the rate limiting by setting the unique ban values
set URIbTime [table timeout -subtable URIBanZ -remaining $b_ip ]
# Set IP/URI Ban count
set URIbCnt [table lookup -notouch -subtable URIbCnt $b_ip ]
# Check for IP/URI ban time
if { $URIbTime > 0 } {
if { $URIbCnt == 1 } {
# If first ban... Reset Ban counter for IP/URI for repeat
failures during blocking period to create a rolling blocking scenario so
block does not start until IP stops.
table set -subtable URIBanZ $b_ip 1 $StdB $StdB
table set -subtable URIbCnt $b_ip 1 $STime $STime
if { $URIbTime <= [expr $StdB - 3600 ] } {
if { $DEBUG_LEVEL >= 1 } {log local0.
"URI_ROLLING_BAN: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip:
$b_ip Banned for: $StdB - Remaining time was: $URIbTime"}
}
}
if { $URIbCnt >= 2 } {
# If already banned... Reset Ban counter for IP for repeat
failures during blocking period to create a rolling blocking scenario so
block does not start until IP stops.
table set -subtable URIBanZ $b_ip 3 $LongB $LongB
table set -subtable URIbCnt $b_ip 3 $LTime $LTime
if { $URIbTime <= [expr $StdB - 3600 ] } {
if { $DEBUG_LEVEL >= 1 } {log local0.
"URI_ROLLING_BAN: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip:
$b_ip Banned for: $StdB - Remaining time was: $URIbTime"}
}
}
# Ensure IP is not in request counter to keep cleaner table.
table delete -subtable URIReqMon $b_ip
# # Log event with remaining time for IP/URI. Not useful with
Rolling ban.
# #if { $DEBUG_LEVEL >= 2 } {log local0. "BANNED_URI: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip: $b_ip Banned for:
$URIbTime, remaining time: [table timeout -subtable URIBanZ -remaining
$b_ip ]"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# Send 403 page and reject. HTML content and 403 response.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?rs=yes"
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
###########################################################################
# Set R8L Counters for R8L URIs
set URIreqCnt [table lookup -notouch -subtable URIReqMon $b_ip ]
# Log Request Counters
if { $DEBUG_LEVEL >= 1 } {log local0. "BAIU_R8LURI_REQCNT:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ URIReqCnt:$URIreqCnt
MonIP: $b_ip "}
###########################################################################
# Check if IP exists in URI table if not create entry otherwise
increment
if { $URIreqCnt == "" } {
# set the lifetime timeout value so request will go away
after a set amount of time
set URIreqCnt [ table set -subtable URIReqMon $b_ip 1
$STime $STime ]
} else {
# Increment URI Counter
set URIreqCnt [ table incr -notouch -subtable URIReqMon
$b_ip ]
}
###########################################################################
# Set URI table limit and let connections pass if table is full
if { $URICnt >= 100000} {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_TABLE_FULL: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ URICnt=$URICnt - They shall pass
"}
# Delete latest IP from URIReqMon table to free up new spot.
table delete -subtable URIReqMon $b_ip
return
} else {
if { $URIbCnt >= 100000} {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_BAN_TABLE_FULL:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ URIbCnt=$URIbCnt - They
shall pass "}
return
}
}
###########################################################################
#
# Check if too many request for a URI
if { $URIreqCnt > $URIMaxReqs } {
# Set the short ban time and remove httpRequests
set URIbTime $StdB
table delete -subtable URIReqMon $b_ip
# Check IP ban levels and set ban times
if { $URIbCnt == "" } {
table set -subtable URIbCnt $b_ip 1 $LTime $LTime
} else {
set URIbCnt [table incr -notouch -subtable URIbCnt $b_ip ]
# Check if we need to ban them for longer
if { $URIbCnt > $MaxBs } {
set URIbTime $LongB
}
}
# Ban them for $URIbTime
table set -subtable URIBanZ $b_ip 1 $URIbTime $URIbTime
# #table set -subtable $URI_MONITOR_TBL "$b_ip" "UID: $waf_uid; Ban:
$URIbTime secs" $URIbTime
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_URI: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_ip: $b_ip Banned for:
$URIbTime"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# # HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?ws=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
# DEBUG INFO
if { $DEBUG_LEVEL >= 3 } {
log local0.alert "URIreqCnt: [table lookup -notouch -
subtable URIReqMon $b_ip ]"
log local0.alert "URIbCnt: [table lookup -notouch -subtable
URIbCnt $b_ip]"
log local0.alert "URIbTime: [table lookup -notouch -
subtable URIBanZ $b_ip]"
# #log local0.alert "URI_limit_monitor tbl keys: [table keys
-subtable $URI_MONITOR_TBL -notouch]"
}
return
}
}
when HTTP_REQUEST_DATA priority 51 {
if { $loginuri == 1 } {
###########################################################################
# Set Normal IP & UID Counters
set IPCnt [table keys -subtable "IPReqMon" -count]
set UIDCnt [table keys -subtable "UIDReqMon" -count]
###########################################################################
# Begin the rate limiting by setting the unique ban values
set IPbTime [table timeout -subtable IPBanZ -remaining $b_ip ]
# Set IP Ban count
set IPbCnt [table lookup -notouch -subtable IPbCnt $b_ip ]
###########################################################################
# Check for IP ban time
if { $IPbTime > 0 } {
if { $IPbCnt == 1 } {
# If first ban... Reset Ban counter for IP for repeat
failures during blocking period to create a rolling blocking scenario so
block does not start until IP stops.
table set -subtable IPBanZ $b_ip 1 $StdB $StdB
table set -subtable IPbCnt $b_ip 1 $STime $STime
if { $IPbTime <= [expr $StdB - 3600 ] } {
if { $DEBUG_LEVEL >= 1 } {log local0.
"IP_ROLLING_BAN: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip:
$b_ip Banned for: $StdB - Remaining time was: $IPbTime"}
}
}
if { $IPbCnt >= 2 } {
# If already banned... Reset Ban counter for IP for repeat
failures during blocking period to create a rolling blocking scenario so
block does not start until IP stops.
table set -subtable IPBanZ $b_ip 3 $LongB $LongB
table set -subtable IPbCnt $b_ip 3 $LTime $LTime
if { $IPbTime <= [expr $StdB - 3600 ] } {
if { $DEBUG_LEVEL >= 1 } {log local0.
"IP_ROLLING_BAN: IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip:
$b_ip Banned for: $StdB - Remaining time was: $IPbTime"}
}
}
# Ensure IP is not in request counter to keep cleaner table.
table delete -subtable IPReqMon $b_ip
# Clear table of tracking entry since IP &or UID are now banned.
!!!Disabling UID since I want this done only at the UID level to be
effective.
table delete -all -subtable I2U$b_ip
# #table delete -all -subtable U2I$b_uid
# ## Ban UID used during IP block invocation. This can be very
dangerous since it would block all UIDs used in a list which will likely
have a lot of good accounts. They could never stop and ban every user.
# #table set -subtable UIDBanZ $b_uid 1 $HalfB $HalfB
# # if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_IP: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_uid: $b_uid Banned
for: $HalfB"}
# # Log event with remaining time for IP. Not useful with
Rolling ban.
# #if { $DEBUG_LEVEL >= 2 } {log local0. "BANNED_IP: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip: $b_ip Banned for: $IPbTime,
remaining time: [table timeout -subtable IPBanZ -remaining $b_ip ]"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
###########################################################################
# Set IP table limit and let connections pass if table is full
if { $IPCnt >= 85000} {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_TABLE_FULL: IP: $tip SID:
$sid~$jsid UID: $waf_uid URI: $uri ~ IPCnt=$IPCnt - They shall pass "}
# Delete latest IP from IPReqMon table to free up new spot.
table delete -subtable IPReqMon $b_ip
table delete -all -subtable I2U$b_ip
return
} else {
if { $IPbCnt >= 85000} {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_BAN_TABLE_FULL:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ IPbCnt=$IPbCnt - They
shall pass "}
return
}
}
###########################################################################
# SetUID Nicest 1 liner. Check length 6 or more, lower case, remove
e-mail, strip special characters, and reduce tracked length to 14.
set uid_nicest [string range [string map {" " ""} [string map {\x21
\x20 \x22 \x20 \x23 \x20 \x24 \x20 \x25 \x20 \x27 \x20 \x28 \x20 \x29 \x20
\x2A \x20 \x2B \x20 \x2C \x20 \x2D \x20 \x2E \x20 \x2F \x20 \x3A \x20 \x3B
\x20 \x3C \x20 \x3D \x20 \x3E \x20 \x3F \x20 \x5B \x20 \x5C \x20 \x5D \x20
\x5E \x20 \x5F \x20 \x60 \x20 \x7B \x20 \x7C \x20 \x7D \x20 \x7E \x20}
[getfield [string map {"%40" "@"} [string trim [string tolower $waf_uid ] ]
] "@" 1 ] ] ] 0 13 ]
# # Trim UID of non print chars and lower case to simplified UID
# #set uid_nice [string trim [string tolower $waf_uid ] ]
# #if string range less than 6 chars do not track return since it's
not min user ID length. Blocking is optional.
# #if { [ string length $uid_nice ] <= 5 } {
# # if { $DEBUG_LEVEL >= 2 } {log local0. "UID_2SHORT: IP: $tip
UID: $uid_nicest SID: $sid~$jsid URI: $uri ~ UID too short" }
# #return
# #}
#
# # Remove e-mail since you can't sign in w/ email ID and then trim
unused characters (not &)(Trims dot dash and underscore too! Still better
for UID simplification)
# #set uid_nicer [string map {"\x20" ""} [string map {\x21 \x20 \x22
\x20 \x23 \x20 \x24 \x20 \x25 \x20 \x27 \x20 \x28 \x20 \x29 \x20 \x2A \x20
\x2B \x20 \x2C \x20 \x2D \x20 \x2E \x20 \x2F \x20 \x3A \x20 \x3B \x20 \x3C
\x20 \x3D \x20 \x3E \x20 \x3F \x20 \x5B \x20 \x5C \x20 \x5D \x20 \x5E \x20
\x5F \x20 \x60 \x20 \x7B \x20 \x7C \x20 \x7D \x20 \x7E \x20} [getfield
[string map {"%40" "@"} $uid_nice ] "@" 1 ] ] ]
# # Use only first 14 characters of UID
# #set uid_nicest [string range $uid_nicer 0 13]
# ## Do not track e-mail addresses with UIDs. Useful with UID Nicest
1 liner that doesn't strip e-mails when e-mails aren't considered as UIDs
# #set uid_nice [string map {"%40" "@"} $waf_uid ]
# #if { $uid_nice contains "@" } {
# #if { $DEBUG_LEVEL >= 2 } {log local0. "NON_UID: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ UID is e-mail - do not track
UID-Nice:$uid_nice" }
# #return
# #}
# ## SetUID Nicest 1 liner but not using e-mail. Check length 6 or
more, lower case, strip special characters, and reduce tracked length to
14. Combine with e-mail dropper above.
# #set uid_nicest [string range [string map {" " ""} [string map
{\x21 \x20 \x22 \x20 \x23 \x20 \x24 \x20 \x25 \x20 \x27 \x20 \x28 \x20 \x29
\x20 \x2A \x20 \x2B \x20 \x2C \x20 \x2D \x20 \x2E \x20 \x2F \x20 \x3A \x20
\x3B \x20 \x3C \x20 \x3D \x20 \x3E \x20 \x3F \x20 \x5B \x20 \x5C \x20 \x5D
\x20 \x5E \x20 \x5F \x20 \x60 \x20 \x7B \x20 \x7C \x20 \x7D \x20 \x7E \x20}
[string trim [string tolower $uid_nice ] ] ] ] 0 13 ]
# Check after shortening tracking UID is still minimum length. If
not then return or drop. my.uid is 6 but wld be myuid n then be dropped if
dot is char in uid_nicest logic.
if { [ string length $uid_nicest ] <= 5 } {
if { $DEBUG_LEVEL >= 2 } {log local0. "UID_2SHORT: IP: $tip
UID: $uid_nicest SID: $sid~$jsid URI: $uri ~ UID still too short" }
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
#HTTP::respond 403 content {<html>blocked</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
#reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
#event disable all
return
}
###########################################################################
# This line builds the Ban UID value
set b_uid "b_$uid_nicest"
###########################################################################
# Set UID Ban Time
set UIDbTime [table timeout -subtable UIDBanZ -remaining $b_uid ]
# Set UID Ban count
set UIDbCnt [table lookup -notouch -subtable UIDbCnt $b_uid ]
# Set UID table limit and let connections pass if table is full
if { $UIDCnt >= 100000} {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_TABLE_FULL: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ UIDCnt=$UIDCnt - They shall pass
"}
# Delete latest UID from UIDReqMon table to free up new spot.
table delete -subtable UIDReqMon $b_uid
table delete -all -subtable I2U$b_ip
table delete -all -subtable U2I$b_uid
return
} else {
if { $UIDbCnt >= 100000 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_BAN_TABLE_FULL:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ UIDbCnt=$UIDbCnt - They
shall pass "}
return
}
}
###########################################################################
# Check if UID is white listed. If so let them pass. You must use
simplified UIDs for whitelisting
# We use this one here because we still want an IP Req Count.
if { ( [class match $waf_uid equals R8L_GOOD_UID ] ) } {
if { $DEBUG_LEVEL >= 2 } {log local0. "WHITELIST_UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Username Whitelisted"}
return
}
###########################################################################
# Check for UID ban time
if { $UIDbTime > 0 } {
## Reset Ban counter for UID for repeat failures during blocking
period to create a rolling blocking scenario so block does not start until
UID stops.
# # UID rolling ban VERY helpful to block list attackers. Still can
ban valid account.
# #table set -subtable UIDBanZ $b_uid 1 $HalfB $HalfB
# #if { $IPbTime <= [expr $StdB - 3600 ] } {
# #if { $DEBUG_LEVEL >= 1 } {log local0. "UID_ROLLING_BAN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_uid: $b_uid Banned
for: $HalfB - - Remaining time was: $UIDbTime""}
# #}
# Ensure UID is not in request counter to keep cleaner table.
table delete -subtable UIDReqMon $b_uid
# Clear table of tracking entry since IP &or UID are now banned.
!!! I use both and just IP for IPBans to be the most effective and keep the
cleanest tables.
table delete -all -subtable I2U$b_ip
table delete -all -subtable U2I$b_uid
## Ban IP used during UID block invocation
table set -subtable IPBanZ $b_ip 1 $StdB $StdB
table set -subtable IPbCnt $b_ip 1 $STime $STime
# #if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_ip: $b_ip Banned
for: $StdB"}
# Log event with remaining time for UID. Not useful with
Rolling UID ban.
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ b_ip: $b_ip Banned for: $StdB, -
UID Ban Time Remaining: $UIDbTime"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
###########################################################################
# Set IP & UID Request Counters for Login URI
set IPreqCnt [table lookup -notouch -subtable IPReqMon $b_ip ]
set UIDreqCnt [table lookup -notouch -subtable UIDReqMon $b_uid ]
# Log Request Counters. Useful for investigating behavior
and tracking all UIDs at the same time.
if { $DEBUG_LEVEL >= 1 } {log local0. "BAIU_LOGIN_REQCNT:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ IPReqCnt:$IPreqCnt
UIDReqCnt:$UIDreqCnt MonIP: $b_ip MonUID: $b_uid "}
###########################################################################
# Check if IP exists if not create entry otherwise increment
if { $IPreqCnt == "" } {
# set the lifetime timeout value so request will go away
after a set amount of time
set IPreqCnt [ table set -subtable IPReqMon $b_ip 1 $STime
$STime ]
} else {
# Increment IP Counter
set IPreqCnt [ table incr -notouch -subtable IPReqMon $b_ip
]
}
###########################################################################
# Check if UID exists if not create entry otherwise increment
if { $UIDreqCnt == "" } {
# set the lifetime timeout value so request will go away
after a set amount of time
set UIDreqCnt [ table set -subtable UIDReqMon $b_uid 1
$STime $STime ]
} else {
# Increment UID Counter
set UIDreqCnt [ table incr -notouch -subtable UIDReqMon
$b_uid ]
}
###########################################################################
#
# ... Sword - Single IP/UID Tracking/Banning
###########################################################################
#
###########################################################################
#
# /// Hydra - Single to Many IP/UID Tracking/Banning
###########################################################################
#
# Single IP tracking all UIDs - Set values for single IP calling to
multi UIDs
set I2UuReq [ table set -subtable I2U$b_ip $b_uid 1 $STime $STime ]
# Single UID tracking all IPs - Set values for all IPs calling to
UID
set U2IuReq [ table set -subtable U2I$b_uid $b_ip 1 $STime $STime ]
###########################################################################
#
# ... Hydra - Single to Many IP/UID Tracking/Banning
###########################################################################
#
###########################################################################
#
# /// Sword - Single IP/UID Tracking/Banning
###########################################################################
#
# Check if too many request for an IP
if { $IPreqCnt > $IPMaxReqs } {
# Set the short ban time and remove httpRequests
set IPbTime $StdB
set UIDbTime $HalfB
table delete -subtable IPReqMon $b_ip
# Check IP ban levels and set ban times
if { $IPbCnt == "" } {
table set -subtable IPbCnt $b_ip 1 $LTime $LTime
table set -subtable UIDbCnt $b_uid 1 $STime $STime
} else {
set IPbCnt [table incr -notouch -subtable IPbCnt $b_ip ]
table set -subtable UIDbCnt $b_uid 1 $STime $STime
# Check if we need to ban them for longer
if { $IPbCnt >= $MaxBs } {
set IPbTime $LongB
set UIDbTime $HalfB
}
}
# Ban them for $IPbTime or other - UID set to half to not punish
actual user but still block. If actual user then will get custom block
page to call in and alert of fraud.
table set -subtable IPBanZ $b_ip 1 $IPbTime $IPbTime
table set -subtable UIDBanZ $b_uid 1 $UIDbTime $UIDbTime
# Clean up Hydra Tables
table delete -all -subtable I2U$b_ip
# #table delete -all -subtable U2I$b_uid
# #table set -subtable $IP_MONITOR_TBL "$b_ip" "UID: $waf_uid; Ban:
$IPbTime secs" $IPbTime
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_IP: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_ip: $b_ip Banned for:
$IPbTime"}
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_IP: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_uid: $b_uid Banned
for: $UIDbTime"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
# DEBUG INFO
if { $DEBUG_LEVEL >= 3 } {
log local0.alert "IPreqCnt: [table lookup -notouch -
subtable IPReqMon $b_ip ]"
log local0.alert "IPbCnt: [table lookup -notouch -subtable
IPbCnt $b_ip]"
log local0.alert "IPbTime: [table lookup -notouch -subtable
IPBanZ $b_ip]"
# #log local0.alert "IP_limit_monitor tbl keys: [table keys
-subtable $IP_MONITOR_TBL -notouch]"
}
###########################################################################
# Check if too many requests for a user ID
if { $UIDreqCnt > $UIDMaxReqs } {
# Set the short ban time and remove httpRequests
set IPbTime $StdB
set UIDbTime $HalfB
table delete -subtable UIDReqMon $b_uid
# Check UID ban levels and set ban times
if { $UIDbCnt == "" } {
table set -subtable IPbCnt $b_ip 1 $STime $STime
table set -subtable UIDbCnt $b_uid 1 $LTime $LTime
} else {
table set -subtable IPbCnt $b_ip 1 $STime $STime
set UIDbCnt [table incr -notouch -subtable UIDbCnt $b_uid]
# Check if we need to ban them for longer
if { $UIDbCnt > $MaxBs } {
set IPbTime $StdB
set UIDbTime $StdB
}
}
# Ban them for $***bTime
table set -subtable IPBanZ $b_ip 1 $IPbTime $IPbTime
table set -subtable UIDBanZ $b_uid 1 $UIDbTime $UIDbTime
# Clean up Hydra Tables
table delete -all -subtable I2U$b_ip
table delete -all -subtable U2I$b_uid
# #table set -subtable $UID_MONITOR_TBL "$b_uid" "UID: $waf_uid;
Ban:$UIDbTime secs" $UIDbTime
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_ip: $b_ip Banned for:
$IPbTime"}
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ BANNING: b_uid: $b_uid Banned
for: $UIDbTime"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
# DEBUG INFO
if { $DEBUG_LEVEL >= 3 } {
log local0.alert "UIDreqCnt: [table lookup -notouch -
subtable UIDReqMon $b_uid ]"
log local0.alert "UIDbCnt: [table lookup -notouch -subtable
UIDbCnt $b_uid]"
log local0.alert "UIDbTime: [table lookup -notouch -
subtable UIDBanZ $b_uid]"
# #log local0.alert "UID_limit_monitor tbl keys: [table keys
-subtable $UID_MONITOR_TBL -notouch]"
}
###########################################################################
# ... Sword - Single IP/UID Tracking/Banning
###########################################################################
###########################################################################
# /// Hydra - Single to Many IP/UID Tracking/Banning - - Multi-Head
Attack
###########################################################################
# Set Single to many IPs/UIDs Counters
set I2UCnt [table keys -subtable I2U$b_ip -count ]
set U2ICnt [table keys -subtable U2I$b_uid -count ]
if { $DEBUG_LEVEL >= 1 } {log local0. "HYDRA_HEAD_CNT: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ I2UCnt:$I2UCnt U2ICnt:$U2ICnt
b_ip: $b_ip b_uid: $b_uid"}
###########################################################################
#
# Check if too many unique IPs have called a single UID
# Uncomment one or both logs. Last 1 is best.
if { $I2UCnt >= 17 } {
set IPbTime $StdB
set UIDbTime $HalfB
#
###########################################################################
#
# Begin banning ALL (foreach) unique UIDs from single IP
# foreach I2U_uid [table keys -subtable I2U$b_ip ] ### Put start
brace here if using foreach!!!!
#
###########################################################################
#
# # Max Request Counts for IP calling multiple UIDs
# #table set -subtable IPReqMon $b_ip 5000 $STime $STime
# Ban IP calling multiple UIDs !!! Disabling to use set max request
counts instead.
table set -subtable IPBanZ $b_ip 1 $IPbTime $IPbTime
table set -subtable IPbCnt $b_ip 1 $IPbTime $IPbTime
# ## Ban all UIDs the single IP used. Could ban a lot of good
accounts.
# #table set -subtable UIDBanZ $I2U_uid 1 $UIDbTime $UIDbTime
# ## Log item when blocking all UIDs used.
# #if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_IP2UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ 2Many UIDs for 1 IP Banned
b_ip: $b_ip for: $IPbTime & b_uid: $I2U_uid for: $UIDbTime"}
######### Put end brace here if using foreach!!!!!!!!!! iRule will
not load if you put brace incomplete string despite comment on end
# Ban UID at time of ban.
table set -subtable UIDBanZ $b_uid 1 $UIDbTime $UIDbTime
# Clear table of tracking entry since IP & UID are now banned.
table delete -all -subtable I2U$b_ip
table delete -all -subtable U2I$b_uid
# Clear Request monitor for UID & IP
table delete -subtable IPReqMon $b_ip
table delete -subtable UIDReqMon $b_uid
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_IP2UID: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ 2Many UIDs from 1 IP Banned
b_ip: $b_ip for: $IPbTime & b_uid: $b_uid for: $UIDbTime"}
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
###########################################################################
#
# Check if too many unique UIDs have been called from a single IP
# Uncomment one or both logs. Second is best.
if { $U2ICnt >= 8 } {
set IPbTime $StdB
set UIDbTime $HalfB
###########################################################################
#
# Begin banning ALL (foreach) unique IPs that called single UID
foreach U2I_tip [table keys -subtable U2I$b_uid ] {
###########################################################################
#
# Ban all IPs used to call single UID
table set -subtable IPBanZ $U2I_tip 1 $IPbTime $IPbTime
table set -subtable IPbCnt $U2I_tip 1 $IPbTime $IPbTime
# Max request counts & reset timers for all IPs used to call single
UID
#table set -subtable IPReqMon $U2I_tip 5000 $STime $STime
# ## Standard log. Use 2nd if banning all IPs used for single
UID.
# #if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID2IP: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ 2Many IPs for 1 UID - Banned
b_ip: $b_ip for: $IPbTime & b_uid: $b_uid for: $UIDbTime"}
# Log item when blocking all IPs used.
if { $DEBUG_LEVEL >= 1 } {log local0. "BANNED_UID2IP: IP:
$U2I_tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ 2Many IPs for 1 UID -
Last IP: $tip - Banned b_ip: $U2I_tip for: $IPbTime & b_uid: $b_uid for:
$UIDbTime"}
# ## Clear table of tracking entry since IPs are now banned. !!!
Disabling to use set max request counts instead.
# #table delete -all -subtable I2U$U2I_tip
# ## Clear Request monitor for all IPs. !!! Disabling to use set max
request counts instead.
# #table delete -subtable IPReqMon $U2I_tip
}
# Clear table of tracking entry since UID is now banned. Put
outside of (foreach) so command isn't repeated unnecessarily.
table delete -all -subtable U2I$b_uid
# Clear Request monitor for UID
table delete -subtable UIDReqMon $b_uid
# Ban UID called from multiple IPs
table set -subtable UIDBanZ $b_uid 1 $UIDbTime $UIDbTime
# Drop packets or send to 403 and reject rest. Uncomment only one
response set at a time.
# HTTP::respond 302 Location
"http://www.companyabc.com/notavail/en/403.html?id=yes"
# Send 403 page and reject. HTML content and 403 response.
HTTP::respond 403 content {<html> blocked </html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
# Drop packets and do not respond (504 no response from server)
#drop
# disable events for this connection
event disable all
return
}
###########################################################################
# ... Hydra - Single to Many IP/UID Tracking - Multi-Head Attack
###########################################################################
###########################################################################
# /// Torch - Request Multiplier - Burn Them Down
###########################################################################
# This section defines the attributes to be used with the $R8X
multiplier for faster banning
# Null values to identify which page has been called. These are
used in the HTTP response to determine if a failed login attempt was made.
If so the request count will add 1 doubling the count making them fail
faster.
set lp1 0
set lp2 0
set lp3 0
set lp4 0
###########################################################################
# Set the URIs to be used. We use lp values since we track on Login
Pages.
if { $uri equals "/ires/en-US/html/SignInConfirmation" } {
set lp1 1
}
if { $uri equals "/ires/en-US/html/SignInFlyout" } {
set lp2 1
}
if { $uri equals "/login-online-profile" } {
set lp3 1
}
if { $uri equals "/cpacct" } {
set lp4 1
}
}
}
when HTTP_RESPONSE priority 53 {
###########################################################################
# Check to see if a tracked URI is called. We use focus on Logins
so we use the Login DG.
if { ( not [class match $uri equals XXX_LOGIN_URI ] ) } {
return
}
# Check which login page was used and then determine if the login
failed or not. If failure, increase request counter by $R8X value
# This is the login page for SignInConfirmation on CH.com
if { $lp1 == 1 } {
if { [HTTP::cookie exists "guestdata" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Multiplying Request
Count $R8X times"}
}
}
# This is the login page for SignInConfirmation on CH.com
if { $lp1 == 1 } {
if { [HTTP::cookie exists "sid" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Multiplying Request
Count $R8X times"}
}
}
# This is the login page for SignInFlyout on CH.com
if { $lp2 == 1 } {
if { [HTTP::cookie exists "guestdata" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Multiplying Request
Count $R8X times"}
}
}
# This is the login page for SignInFlyout on CH.com
if { $lp2 == 1 } {
if { [HTTP::cookie exists "sid" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Multiplying Request
Count $R8X times"}
}
}
# This is the login page for IWS
if { $lp3 == 1 } {
if { [HTTP::cookie exists "iwsLoggedIn" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri - Multiplying Request
Count $R8X times"}
}
}
# This is the login page for Mobile. This is also the landing page
of the sign in page but CPID cookie is always there.
if { $lp4 == 1 } {
if { [HTTP::cookie exists "cpid" ] != 1 } {
table incr -notouch -subtable IPReqMon $b_ip $R8X
table incr -notouch -subtable UIDReqMon $b_uid $R8X
if { $DEBUG_LEVEL >= 2 } {log local0. "BAD_LOGIN:
IP: $tip SID: $sid~$jsid UID: $waf_uid URI: $uri - Multiplying Request
Count $R8X times"}
} else {
# Delete Hydra entries due to successful auth.
table delete -subtable I2U$b_ip $b_uid
table delete -subtable U2I$b_uid $b_ip
}
}
###########################################################################
# ... Torch - Request Multiplier - Burn Them Down
###########################################################################
}
==XXX_071_BAIU_CnC.txt==
# This iRule drops items from the BAIU Rate Limiter tables
#
#
when HTTP_REQUEST priority 70 {
# If not from corporate LAN or more specific network, then do not
invoke rule.
if { (not [class match $tip equals XXX_LAN ] ) } {
return
}
# Set URI values to 0
set del_uid 0
set del_uid_all 0
set del_ip 0
set del_ip_all 0
set del_uri 0
set del_uri_all 0
set del_I2U 0
set del_U2I 0
set add_ip 0
set add_uid 0
set add_uri 0
set add_I2U 0
set add_U2I 0
set nud_ip 0
set nud_uid 0
set nud_uri 0
# Set Ban times
set StdB 172800
set HalfB 43200
# Set value for Kill request count
set KillReq 5000
# Set value for IP/UID Nudge request count
set IUNudReq 10
# Set value for URI Nudge request count
set URINudReq 100
###########################################################################
#
# enable HTTP Query value for drop URIs. Can use be used for future
use too.
set uri_q [string range [HTTP::query] 0 34 ]
###########################################################################
#
# Delete IP/UID/URI Section
###########################################################################
#
# Set value of item to delete from table based on query
if { ( $uri equals "/RATE_LIMITER_DROP_IP" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_IP_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_ip 1
set uri_q [string range [HTTP::query] 0 17 ]
}
# set purge full IP table value if purge table URI was called with
proper -all query
if { ( $uri equals "/RATE_LIMITER_DROP_IP/DR0P4LL" ) and ( [class
match $tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_IP_ALL: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_ip_all 1
set uri_q [string range [HTTP::query] 0 4 ]
}
# Set value of item to delete from table based on query
if { ( $uri equals "/RATE_LIMITER_DROP_UID" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_UID_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_uid 1
set uri_q [string range [HTTP::query] 0 15 ]
}
# Set purge full UID table value if purge table URI was called with
proper -all query
if { ( $uri equals "/RATE_LIMITER_DROP_UID/DR0P4LL" ) and ( [class
match $tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_UID_ALL: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query:
$uri_q"}
set del_uid_all 1
set uri_q [string range [HTTP::query] 0 3 ]
}
if { ( $uri equals "/RATE_LIMITER_DROP_URI" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_URI_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_uri 1
set uri_q [string range [HTTP::query] 0 17 ]
}
# set purge full IP table value if purge table URI was called with
proper -all query
if { ( $uri equals "/RATE_LIMITER_DROP_URI/DR0P4LL" ) and ( [class
match $tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_URI_ALL: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query:
$uri_q"}
set del_uri_all 1
set uri_q [string range [HTTP::query] 0 4 ]
}
###########################################################################
#
# Set value of item to delete from table based on query
if { ( $uri equals "/RATE_LIMITER_DROP_I2U" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_I2U_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_I2U 1
set split_q [split [HTTP::query] "&"]
for {set i 0} {$i < [llength $split_q]} {incr i} {
set qip [lindex $split_q 0]
set quid [lindex $split_q 1]
}
}
# Set value of item to delete from table based on query
if { ( $uri equals "/RATE_LIMITER_DROP_U2I" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "DEL_U2I_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Delete URI called Query: $uri_q"}
set del_U2I 1
set split_q [split [HTTP::query] "&"]
for {set i 0} {$i < [llength $split_q]} {incr i} {
set qip [lindex $split_q 0]
set quid [lindex $split_q 1]
}
}
###########################################################################
#
# Add IP/UID/URI Section
###########################################################################
#
# Set value of IP to add to the IP table based on query
if { ( $uri equals "/RATE_LIMITER_ADD_IP" ) and ( [class match $tip
equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "ADD_IP_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set add_ip 1
set uri_q [string range [HTTP::query] 0 17 ]
}
# Set value of IP to add to the UID table based on query
if { ( $uri equals "/RATE_LIMITER_ADD_UID" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "ADD_UID_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set add_uid 1
set uri_q [string range [HTTP::query] 0 15 ]
}
# Set value of IP to add to the URI table based on query
if { ( $uri equals "/RATE_LIMITER_ADD_URI" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "ADD_URI_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set add_uri 1
set uri_q [string range [HTTP::query] 0 15 ]
}
###########################################################################
#
# Set value of IP & UID to add to the I2U table based on query
if { ( $uri equals "/RATE_LIMITER_ADD_I2U" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "ADD_I2U_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set add_I2U 1
set split_q [split [HTTP::query] "&"]
for {set i 0} {$i < [llength $split_q]} {incr i} {
set qip [lindex $split_q 0]
set quid [lindex $split_q 1]
}
}
# Set value of IP & UID to add to the U2I table based on query
if { ( $uri equals "/RATE_LIMITER_ADD_U2I" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "ADD_U2I_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set add_U2I 1
set split_q [split [HTTP::query] "&"]
for {set i 0} {$i < [llength $split_q]} {incr i} {
set qip [lindex $split_q 0]
set quid [lindex $split_q 1]
}
}
###########################################################################
#
# Nudge URI Section
###########################################################################
#
# Set value of IP to nudge to the IP table based on query
if { ( $uri equals "/RATE_LIMITER_NUDGE_IP" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "NUDG3_IP_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query: $uri_q"}
set nud_ip 1
set uri_q [string range [HTTP::query] 0 17 ]
}
# Set value of IP to nudge to the UID table based on query
if { ( $uri equals "/RATE_LIMITER_NUDGE_UID" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "NUDG3_UID_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query:
$uri_q"}
set nud_uid 1
set uri_q [string range [HTTP::query] 0 15 ]
}
# Set value of IP to nudge to the URI table based on query
if { ( $uri equals "/RATE_LIMITER_NUDGE_URI" ) and ( [class match
$tip equals XXX_LAN ] ) } {
if { $DEBUG_LEVEL >= 1 } {log local0. "NUDG3_URI_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Add URI called Query:
$uri_q"}
set nud_uri 1
set uri_q [string range [HTTP::query] 0 15 ]
}
###########################################################################
#
# Process Section
###########################################################################
#
# Begin Table deletion processes
###########################################################################
#
# Used for IP Table deletion of entries
# Delete value extracted from query if delete URI is called
if { $del_ip == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_DROPTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Request: [string range
$uri_q 0 15 ] - dropping all IP elements for Request: [string range $uri_q
0 17 ]"}
# Delete item from table if IP request was made
table delete -subtable IPBanZ $uri_q
table delete -subtable IPReqMon $uri_q
table delete -subtable IPbCnt $uri_q
table delete -all -subtable I2U$uri_q
HTTP::respond 403 content {<html>IP Dropped from Ban and
Monitor</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Delete entire table assuming -all was in query
if { $del_ip_all == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_DROPTBL_ALL: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Request: [string range
$uri_q 0 4 ] - dropping IPBanZ,IPReqMon,IPbCnt ALL ELEMENTS"}
# Delete item from table if IP request was made
table delete -all -subtable IPBanZ
table delete -all -subtable IPReqMon
table delete -all -subtable IPbCnt
HTTP::respond 403 content {<html>ALL IP BASED TRACKING ELEMENTS
DROPPED</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for UID Table deletion of entries
# Delete value extracted from query if delete URI is called
if { $del_uid == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_DROPTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ dropping all UID elements
for Request: [string range $uri_q 0 15 ]"}
# Delete item from table if UID request was made
table delete -subtable UIDBanZ $uri_q
table delete -subtable UIDReqMon $uri_q
table delete -subtable UIDbCnt $uri_q
table delete -all -subtable U2I$uri_q
HTTP::respond 403 content {<html>UID Dropped from Ban and
Monitor</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Delete entire table assuming -all was in query
if { $del_uid_all == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_DROPTBL_ALL: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Request: [string range
$uri_q 0 4 ] - dropping UIDBanZ,UIDReqMon,UIDbCnt ALL ELEMENTS"}
# Delete item from table if UID request was made
table delete -all -subtable UIDBanZ
table delete -all -subtable UIDReqMon
table delete -all -subtable UIDbCnt
HTTP::respond 403 content {<html>ALL UID BASED TRACKING ELEMENTS
DROPPED</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for URI Table deletion of entries
# Delete value extracted from query if delete URI is called
if { $del_uri == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_DROPTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ dropping all URI elements
for Request: [string range $uri_q 0 15 ]"}
# Delete item from table if URI request was made
table delete -subtable URIBanZ $uri_q
table delete -subtable URIReqMon $uri_q
table delete -subtable URIbCnt $uri_q
HTTP::respond 403 content {<html>IP Dropped from URI Ban and
Monitor</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Delete entire table assuming -all was in query
if { $del_uri_all == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_DROPTBL_ALL: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Request: [string range
$uri_q 0 4 ] - dropping URIBanZ,URIReqMon,URIbCnt ALL ELEMENTS"}
# Delete item from table if URI request was made
table delete -all -subtable URIBanZ
table delete -all -subtable URIReqMon
table delete -all -subtable URIbCnt
HTTP::respond 403 content {<html>ALL URI BASED TRACKING ELEMENTS
DROPPED</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
###########################################################################
#
# Used for I2U Table deletion of entries
# Delete value extracted from query if delete URI is called
if { $del_I2U == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "I2U_DROPTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Dropping IP & UID to I2U
Table: IP:$qip UID:quid"}
# Delete item from table if URI request was made
table delete -subtable I2U$qip $quid
HTTP::respond 403 content {<html>IP & UID Dropped from I2U
Table</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for U2I Table deletion of entries
# Delete value extracted from query if delete URI is called
if { $del_U2I == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "U2I_DROPTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Dropping UID & IP to U2I
Table: UID:quid IP:$qip "}
# Delete item from table if URI request was made
table delete -subtable U2I$quid $qip
HTTP::respond 403 content {<html>UID & IP Dropped from U2I
Table</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
###########################################################################
#
# Begin Table addition processes
###########################################################################
#
# Used for IP Table Addition of entries
# Add value extracted from query if add IP is called
if { $add_ip == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_ADDTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Adding IP to Ban Table with
Request: [string range $uri_q 0 17 ]"}
# ## Set UID Ban Count to 1
# #table set -subtable IPBanZ $uri_q 1 $StdB $StdB
# #table set -subtable IPbCnt $uri_q 1 $StdB $StdB
# Set IP Request Counter over Max to block on next request
table set -subtable IPReqMon $uri_q $KillReq 129600 129600
HTTP::respond 403 content {<html>IP Will Be Banned</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for UID Table deletion of entries
# Add value extracted from query if add UID is called
if { $add_uid == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_ADDTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Adding UID to Ban Table with
Request: [string range $uri_q 0 15 ]"}
# ## Set UID Ban Count to 1
# #table set -subtable UIDBanZ $uri_q 1 $HalfB $HalfB
# #table set -subtable UIDbCnt $uri_q 1 $HalfB $HalfB
# Set UID Request Counter over Max to block on next request
table set -subtable UIDReqMon $uri_q $KillReq 129600 129600
HTTP::respond 403 content {<html>UID Will Be Banned</html>}
noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for URI Table deletion of entries
# Add value extracted from query if add URI is called
if { $add_uri == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_ADDTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Adding IP to URI Ban Table
with Request: [string range $uri_q 0 15 ]"}
# ## Set URI Ban Count to 1
# #table set -subtable URIBanZ $uri_q 1 $StdB $StdB
# #table set -subtable URIbCnt $uri_q 1 $StdB $StdB
# Set URI Request Counter over Max to block on next request
table set -subtable URIReqMon $uri_q $KillReq 129600 129600
HTTP::respond 403 content {<html>URI Will Be Banned</html>}
noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
###########################################################################
#
# Used for I2U Table addition of entries
# Add value extracted from query if add URI is called
if { $add_I2U == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "I2U_ADDTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Adding IP & UID to I2U
Table: IP:$qip UID:quid"}
# Set IP Request Counter over Max to block on next request
table set -subtable I2U$qip $quid 1 129600 129600
HTTP::respond 403 content {<html>IP Will Be Added to I2U
Table</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for U2I Table addition of entries
# Add value extracted from query if add URI is called
if { $add_U2I == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "U2I_ADDTBL_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Adding UID & IP to U2I
Table: UID:quid IP:$qip"}
# Set UID Request Counter over Max to block on next request
table set -subtable U2I$quid $qip 1 129600 129600
HTTP::respond 403 content {<html>UID Will Be Added to U2I
Table</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
###########################################################################
#
# Begin Table nudging processes
###########################################################################
#
# Used for IP Table nudging of entries
# Add value extracted from query if add URI is called
if { $nud_ip == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "IP_NUDGE_1: IP: $tip
SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Nudging IP: $nud_ip with ReqCnt:
$IUNudReq"}
# Increment IP Request Counter per nudge value
table incr -notouch -subtable IPReqMon $uri_q $IUNudReq
HTTP::respond 403 content {<html>IP Will Be Nudged</html>} noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for UID Table nudging of entries
# Add value extracted from query if add URI is called
if { $nud_uid == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "UID_NUDGE_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Nudging UID: $nud_uid with
ReqCnt:$IUNudReq"}
# Increment UID Request Counter per nudge value
table incr -notouch -subtable UIDReqMon $uri_q $IUNudReq
HTTP::respond 403 content {<html>UID Will Be Nudged</html>}
noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
# Used for URI Table nudging of entries
# Add value extracted from query if add URI is called
if { $nud_uri == 1 } {
if { $DEBUG_LEVEL >= 1 } {log local0. "URI_NUDGE_1: IP:
$tip SID: $sid~$jsid UID: $waf_uid URI: $uri ~ Nudging IP: $nud_uri with
ReqCnt:$URINudReq"}
# Increment IP Request Counter per nudge value
table incr -notouch -subtable URIReqMon $uri_q $URINudReq
HTTP::respond 403 content {<html>URI IP Will Be Nudged</html>}
noserver
# Send 403 page and reject. Close TCP connection so client can't
make further requests otherwise will succeed (NEEDED for 403 content page)
reject
}
}
Scripts:¶
==rollingban_sync.txt ==
#!/bin/bash
rm /var/log/sync/logs/roll_hr/hourlyrollip
rm /var/log/sync/logs/roll_hr/hourlyrolluid
rm /var/log/sync/logs/roll_hr/hourlyrolluri
rm /var/log/sync/logs/roll_hr/rolledips
rm /var/log/sync/logs/roll_hr/rolleduids
rm /var/log/sync/logs/roll_hr/rolleduris
for ri in /var/log/sync/logs/roll_hr/addip*; do rm $ri; done
for ru in /var/log/sync/logs/roll_hr/adduid*; do rm $ru; done
for rr in /var/log/sync/logs/roll_hr/adduri*; do rm $rr; done
for dallb in /var/log/sync/logs/roll_hr/allrolled*; do rm $dallb; done
hrago=$(date --date="-1 hours" "+%b %_d %H")
now=$(date "+%b %_d %H")
cat /var/log/ltm |grep "$hrago:"| egrep 'IP_ROLLING_BAN'
>>/var/log/sync/logs/roll_hr/hourlyrollip
cat /var/log/ltm |grep "$hrago:"| egrep 'UID_ROLLING_BAN'
>>/var/log/sync/logs/roll_hr/hourlyrolluid
cat /var/log/ltm |grep "$hrago:"| egrep 'URI_ROLLING_BAN'
>>/var/log/sync/logs/roll_hr/hourlyrolluri
cat /var/log/sync/logs/roll_hr/hourlyrollip | egrep 'b_ip: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/roll_hr/allrolledips
cat /var/log/sync/logs/roll_hr/hourlyrolluid | egrep 'b_uid: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/roll_hr/allrolleduids
cat /var/log/sync/logs/roll_hr/hourlyrolluri | egrep 'b_ip: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/roll_hr/allrolleduris
sort /var/log/sync/logs/roll_hr/allrolledips | uniq >
/var/log/sync/logs/roll_hr/rolledips
sort /var/log/sync/logs/roll_hr/allrolleduids | uniq >
/var/log/sync/logs/roll_hr/rolleduids
sort /var/log/sync/logs/roll_hr/allrolleduris | uniq >
/var/log/sync/logs/roll_hr/rolleduris
sed -i 's#b_#GET /RATE_LIMITER_ADD_IP?b_#g'
/var/log/sync/logs/roll_hr/rolledips
sed -i 's#b_#GET /RATE_LIMITER_ADD_UID?b_#g'
/var/log/sync/logs/roll_hr/rolleduids
sed -i 's#b_#GET /RATE_LIMITER_ADD_URI?b_#g'
/var/log/sync/logs/roll_hr/rolleduris
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/roll_hr/rolledips
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/roll_hr/rolleduids
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/roll_hr/rolleduris
split -d -l 1 /var/log/sync/logs/roll_hr/rolledips
/var/log/sync/logs/roll_hr/addip
split -d -l 1 /var/log/sync/logs/roll_hr/rolleduids
/var/log/sync/logs/roll_hr/adduid
split -d -l 1 /var/log/sync/logs/roll_hr/rolleduris
/var/log/sync/logs/roll_hr/adduri
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/roll_hr/addip*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/roll_hr/adduid*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/roll_hr/adduri*
for ri in /var/log/sync/logs/roll_hr/addip*; do nc 192.168.255.200 80 <
$ri; done
for ru in /var/log/sync/logs/roll_hr/adduid*; do nc 192.168.255.200 80 <
$ru; done
for rr in /var/log/sync/logs/roll_hr/adduri*; do nc 192.168.255.200 80 <
$rr; done
==banlist_sync.txt==
rm /var/log/sync/logs/banned/hourlybanip
rm /var/log/sync/logs/banned/hourlybanuid
rm /var/log/sync/logs/banned/hourlybanuri
rm /var/log/sync/logs/banned/hourlydropip
rm /var/log/sync/logs/banned/hourlydropuid
rm /var/log/sync/logs/banned/bannedips
rm /var/log/sync/logs/banned/banneduids
rm /var/log/sync/logs/banned/banneduris
rm /var/log/sync/logs/banned/droppedips
rm /var/log/sync/logs/banned/droppeduids
for ai in /var/log/sync/logs/banned/addip*; do rm $ai; done
for au in /var/log/sync/logs/banned/adduid*; do rm $au; done
for ar in /var/log/sync/logs/banned/adduri*; do rm $ar; done
for di in /var/log/sync/logs/banned/dropip*; do rm $di; done
for du in /var/log/sync/logs/banned/dropuid*; do rm $du; done
for dalld in /var/log/sync/logs/banned/alldropped*; do rm $dalld; done
for dallb in /var/log/sync/logs/banned/allbanned*; do rm $dallb; done
hrago=$(date --date="-1 hours" "+%b %_d %H")
now=$(date "+%b %_d %H")
cat /var/log/ltm |grep "$now:"| egrep 'BANNED_IP'
>>/var/log/sync/logs/banned/hourlybanip
cat /var/log/ltm |grep "$now:"| egrep 'BANNED_UID'
>>/var/log/sync/logs/banned/hourlybanuid
cat /var/log/ltm |grep "$now:"| egrep 'BANNED_URI'
>>/var/log/sync/logs/banned/hourlybanuri
cat /var/log/ltm |grep "$now:"| egrep '_Dr0p_IP'
>>/var/log/sync/logs/banned/hourlydropip
cat /var/log/ltm |grep "$now:"| egrep '_Dr0p_UID'
>>/var/log/sync/logs/banned/hourlydropuid
cat /var/log/sync/logs/banned/hourlybanip | egrep 'b_ip: b_.* ' -o | awk -F
' ' '{print $2}' >>/var/log/sync/logs/banned/allbannedips
cat /var/log/sync/logs/banned/hourlybanip | egrep 'b_uid: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/banned/allbanneduids
cat /var/log/sync/logs/banned/hourlybanuid | egrep 'b_ip: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/banned/allbannedips
cat /var/log/sync/logs/banned/hourlybanuid | egrep 'b_uid: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/banned/allbanneduids
cat /var/log/sync/logs/banned/hourlybanuri | egrep 'b_ip: b_.* ' -o | awk
-F ' ' '{print $2}' >>/var/log/sync/logs/banned/allbanneduris
cat /var/log/sync/logs/banned/hourlydropip | egrep 'b_.* ' -o | awk -F ' '
'{print $1}' >>/var/log/sync/logs/banned/alldroppedips
cat /var/log/sync/logs/banned/hourlydropuid | egrep 'b_.* ' -o | awk -F ' '
'{print $1}' >>/var/log/sync/logs/banned/alldroppeduids
sort /var/log/sync/logs/banned/allbannedips | uniq >
/var/log/sync/logs/banned/bannedips
sort /var/log/sync/logs/banned/allbanneduids | uniq >
/var/log/sync/logs/banned/banneduids
sort /var/log/sync/logs/banned/allbanneduris | uniq >
/var/log/sync/logs/banned/banneduris
sort /var/log/sync/logs/banned/alldroppedips | uniq >
/var/log/sync/logs/banned/droppedips
sort /var/log/sync/logs/banned/alldroppeduids | uniq >
/var/log/sync/logs/banned/droppeduids
sed -i 's#b_#GET RATE_LIMITER_ADDIP?b_#g'
/var/log/sync/logs/banned/bannedips
sed -i 's#b_#GET RATE_LIMITER_ADDUID?b_#g'
/var/log/sync/logs/banned/banneduids
sed -i 's#b_#GET RATE_LIMITER_ADDURI?b_#g'
/var/log/sync/logs/banned/banneduris
sed -i 's#b_#GET /RATE_LIMITER_DROP_IP?b_#g'
/var/log/sync/logs/banned/droppedips
sed -i 's#b_#GET /RATE_LIMITER_DROP_UID?b_#g'
/var/log/sync/logs/banned/droppeduids
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/banned/bannedips
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/banned/banneduids
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/banned/banneduris
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/banned/droppedips
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/banned/droppeduids
split -a 4 -d -l 1 /var/log/sync/logs/banned/bannedips
/var/log/sync/logs/banned/addip
split -a 4 -d -l 1 /var/log/sync/logs/banned/banneduids
/var/log/sync/logs/banned/adduid
split -a 4 -d -l 1 /var/log/sync/logs/banned/banneduris
/var/log/sync/logs/banned/adduri
split -a 4 -d -l 1 /var/log/sync/logs/banned/droppedips
/var/log/sync/logs/banned/dropip
split -a 4 -d -l 1 /var/log/sync/logs/banned/droppeduids
/var/log/sync/logs/banned/dropuid
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/banned/addip*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/banned/adduid*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/banned/adduri*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/banned/dropip*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/banned/dropuid*
for ai in /var/log/sync/logs/banned/addip*; do nc 192.168.255.200 80 < $ai;
done
for au in /var/log/sync/logs/banned/adduid*; do nc 192.168.255.200 80 <
$au; done
for ar in /var/log/sync/logs/banned/adduri*; do nc 192.168.255.200 80 <
$ar; done
for di in /var/log/sync/logs/banned/dropip*; do nc 192.168.255.200 80 <
$di; done
for du in /var/log/sync/logs/banned/dropuid*; do nc 192.168.255.200 80 <
$du; done
===reqcnt_hourly_sync.txt===
#!/bin/bash
#rm /var/log/sync/logs/req_hour/hourlyreqip
#rm /var/log/sync/logs/req_hour/hourlyrequid
rm /var/log/sync/logs/req_hour/hourlyrequri
#rm /var/log/sync/logs/req_hour/reqhrips
#rm /var/log/sync/logs/req_hour/reqhruids
rm /var/log/sync/logs/req_hour/reqhruris
#rm /var/log/sync/logs/req_hour/allreqhrips
#rm /var/log/sync/logs/req_hour/allreqhruids
rm /var/log/sync/logs/req_hour/allreqhruris
#for ai in /var/log/sync/logs/req_hour/addreqhrip*; do rm $ai; done
#for au in /var/log/sync/logs/req_hour/addreqhruid*; do rm $au; done
for ar in /var/log/sync/logs/req_hour/addreqhruri*; do rm $ar; done
hrago=$(date --date="-1 hours" "+%b %_d %H")
now=$(date "+%b %_d %H")
#cat /var/log/ltm |grep "$hrago:"| egrep 'IPReqCnt:[2-3][1-2]'
>>/var/log/sync/logs/req_hour/hourlyreqip
#cat /var/log/ltm |grep "$hrago:"| egrep 'UIDReqCnt:[2-3][1-2]'
>>/var/log/sync/logs/req_hour/hourlyrequid
cat /var/log/ltm |grep "$hrago:"| egrep 'URIReqCnt:[1-9]00'
>>/var/log/sync/logs/req_hour/hourlyrequri
#cat /var/log/sync/logs/req_hour/hourlyreqip | egrep 'MonIP: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/req_hour/allreqhrips
#cat /var/log/sync/logs/req_hour/hourlyrequid | egrep 'MonUID: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/req_hour/allreqhruids
cat /var/log/sync/logs/req_hour/hourlyrequri | egrep 'MonURI: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/req_hour/allreqhruris
#sort /var/log/sync/logs/req_hour/allreqhrips | uniq >
/var/log/sync/logs/req_hour/reqhrips
#sort /var/log/sync/logs/req_hour/allreqhruids | uniq >
/var/log/sync/logs/req_hour/reqhruids
sort /var/log/sync/logs/req_hour/allreqhruris | uniq >
/var/log/sync/logs/req_hour/reqhruris
#sed -i 's#b_#GET /WAF/RATE_LIMITER_NUDGE_IP?b_#g'
/var/log/sync/logs/req_hour/reqhrips
#sed -i 's#b_#GET /WAF/RATE_LIMITER_NUDGE_UID?b_#g'
/var/log/sync/logs/req_hour/reqhruids
sed -i 's#b_#GET /WAF/RATE_LIMITER_NUDGE_URI?b_#g'
/var/log/sync/logs/req_hour/reqhruris
#sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/req_hour/reqhrips
#sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/req_hour/reqhruids
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/req_hour/reqhruris
#split -a 3 -d -l 1 /var/log/sync/logs/req_hour/reqhrips
/var/log/sync/logs/req_hour/addreqhrip
#split -a 3 -d -l 1 /var/log/sync/logs/req_hour/reqhruids
/var/log/sync/logs/req_hour/addreqhruid
split -a 3 -d -l 1 /var/log/sync/logs/req_hour/reqhruris
/var/log/sync/logs/req_hour/addreqhruri
#perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.255.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/req_hour/addreqhrip*
#perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.255.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/req_hour/addreqhruid*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.255.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/req_hour/addreqhruri*
#for ai in /var/log/sync/logs/req_hour/addreqhrip*; do nc 192.255.255.200
80 < $ai; done
#for au in /var/log/sync/logs/req_hour/addreqhruid*; do nc 192.255.255.200
80 < $au; done
for ar in /var/log/sync/logs/req_hour/addreqhruri*; do nc 192.255.255.200
80 < $ar; done
===reqcnt_10min_sync.txt===
#!/bin/bash
rm /var/log/sync/logs/req_tenmin/tenminreqip
rm /var/log/sync/logs/req_tenmin/tenminrequid
rm /var/log/sync/logs/req_tenmin/reqtenminips
rm /var/log/sync/logs/req_tenmin/reqtenminuids
rm /var/log/sync/logs/req_tenmin/allreqtenminips
rm /var/log/sync/logs/req_tenmin/allreqtenminuids
for tmai in /var/log/sync/logs/req_tenmin/addreqtenminip*; do rm $tmai;
done
for tmau in /var/log/sync/logs/req_tenmin/addreqtenminuid*; do rm $tmau;
done
hrago=$(date --date="-1 hours" "+%b %_d %H")
now=$(date "+%b %_d %H")
tenago=$(date --date="-10 minutes" "+%b %_d %H:%M")
cat /var/log/ltm |grep "${tenago%?}"| egrep 'IPReqCnt:[2-3][12]'
>>/var/log/sync/logs/req_tenmin/tenminreqip
cat /var/log/ltm |grep "${tenago%?}"| egrep 'UIDReqCnt:[2-3][12]'
>>/var/log/sync/logs/req_tenmin/tenminrequid
cat /var/log/sync/logs/req_tenmin/tenminreqip | egrep 'MonIP: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/req_tenmin/allreqtenminips
cat /var/log/sync/logs/req_tenmin/tenminrequid | egrep 'MonUID: b_.* ' -o |
awk -F ' ' '{print $2}' >>/var/log/sync/logs/req_tenmin/allreqtenminuids
sort /var/log/sync/logs/req_tenmin/allreqtenminips | uniq >
/var/log/sync/logs/req_tenmin/reqtenminips
sort /var/log/sync/logs/req_tenmin/allreqtenminuids | uniq >
/var/log/sync/logs/req_tenmin/reqtenminuids
sed -i 's#b_#GET /RATE_LIMITER_IP?b_#g'
/var/log/sync/logs/req_tenmin/reqtenminips
sed -i 's#b_#GET /RATE_LIMITER_UID?b_#g'
/var/log/sync/logs/req_tenmin/reqtenminuids
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/req_tenmin/reqtenminips
sed -e 's#$# HTTP/1.1#' -i /var/log/sync/logs/req_tenmin/reqtenminuids
split -a 3 -d -l 1 /var/log/sync/logs/req_tenmin/reqtenminips
/var/log/sync/logs/req_tenmin/addreqtenminip
split -a 3 -d -l 1 /var/log/sync/logs/req_tenmin/reqtenminuids
/var/log/sync/logs/req_tenmin/addreqtenminuid
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/req_tenmin/addreqtenminip*
perl -i -e 'while(<>){print $_."Host: user_website.com\nUser-Agent:
Mozilla/5.0\nTrue-Client-IP: 192.168.255.200\n\nEOF\n\n";}'
/var/log/sync/logs/req_tenmin/addreqtenminuid*
for tmai in /var/log/sync/logs/req_tenmin/addreqtenminip*; do nc
192.168.255.200 80 < $tmai; done
for tmau in /var/log/sync/logs/req_tenmin/addreqtenminuid*; do nc
192.168.255.200 80 < $tmau; done
The BIG-IP API Reference documentation contains community-contributed content. F5 does not monitor or control community code contributions. We make no guarantees or warranties regarding the available code, and it may contain errors, defects, bugs, inaccuracies, or security vulnerabilities. Your access to and use of any code available in the BIG-IP API reference guides is solely at your own risk.