How to detect and identify the type of Web Application Firewall (WAF)

What is WAF (web application firewall)

WAF, Web Application Firewall is a program that analyzes requests coming to a web server and, based on rules, filters out those that can be potentially dangerous for a website.

Due to the use of WAF, it is impossible to exploit the vulnerabilities of a website, since the requests with the payload do not reach the web server and the web application, they are discarded by the web application firewall.

Why using WAF does not guarantee website protection

In fact, with WAF (as with many other security solutions), things are not so simple. For WAF to be effective, it must be configured correctly. Some WAFs simply do nothing after installation, since all filtering rules are disabled in them.

From the WAF's point of view, the ideal “protection” is when no request can reach the web server – no clients, no danger. And if you just enable all the WAF rules, the web server can stop working, since almost all web requests will be considered “potentially dangerous”.

Of course, these two extremes do not suit webmasters and they need to find a balance when the website is still up and the WAF is still effective. There are hundreds of WAF solutions, and they all vary widely. Configuring some of them comes down to enabling and disabling filtered threat categories in the web interface. To configure some of them, you need to edit text files in which dangerous requests are described in their own WAF language.

To customize the WAF rules as strictly as possible, you need to activate rules that would prohibit everything except the required types of requests to the web server. To do this, you need, first, to have a good understanding of the operation of the protected website, what it needs and what not; and second, a good understanding of the types of attacks, how they are exploited, and how a custom WAF works. All this is difficult and, as you might guess, this ideal is rare.

This is why a penetration tester still has a chance, even if the site is protected by a web application firewall. But, undoubtedly, WAF always complicates the task of auditing site security.

Different WAFs have different filter sets and are subject to filter bypass techniques to varying degrees. For this reason, you need to be able to determine what kind of web protection is used on the site. This is exactly what this article is about – we will get acquainted with several tools designed to detect WAFs, determine their type (manufacturer) and search for possible ways to bypass WAF filtering.

Instructions for installing the programs in question can be found on the cards of each tool, links to which will also be given.

Several sites for our tests, each of which uses a file firewall:

• wise.com
• www.ebay.com
• admitad.com
• apple.com

WAFW00F

The wafw00f program very quickly and accurately detects the WAF type for the specified site. Of the additional functions, wafw00f has the following:

• scanning a site through a proxy
• support for input and output formats csv, json or text

Using the program is very simple – just specify the domain of the site for which you want to know the WAF:

wafw00f URL

For example:

wafw00f wise.com

The “Number of requests” line shows the number of requests made – just two were enough. As a result, WAF Cloudflare was identified.

If you think that the identification of web application security by fingerprints is inaccurate, then add the -a option, as a result, WAFW00F will not stop after the first match, but will check for all fingerprints:

wafw00f -a wise.com

To send requests through a proxy, use the -p option, after which specify the proxy data – SOCKS and authentication are supported, examples of correctly specified proxy parameters:

• http://hostname:8080
• socks5://hostname:1080
• http://user:pass@hostname:8080

The following command uses the Tor network as a proxy:

sudo systemctl start tor
wafw00f -p socks5://127.0.0.1:9050 admitad.com

Despite the slowness of the Tor network, the identification of the web protection (which turned out to be Wordfence from the manufacturer Defiant) was very fast and required only two requests.

The targets for identifying web protection can be collected into a file. When starting the program, you can specify a file with a list of targets, csv, json or text formats are supported. For csv and json, a column or element named “url” is required. Text file format: 1 URL per line.

Example:

wafw00f -i urls.txt

With the -t option you can specify which WAF you want to search for, this option should be especially useful with -i:

wafw00f -i urls.txt -t 'Cloudflare (Cloudflare Inc.)'

By the way, you can display a complete list of supported web application firewalls with the command:

wafw00f -l

Pay attention in what format the required WAF is indicated. If you have doubts about how to specify the WAF correctly, then go to https://github.com/EnableSecurity/wafw00f/tree/master/wafw00f/plugins, find the file with the WAF you need and open it. Find the line starting with NAME, for example:

NAME = 'ACE XML Gateway (Cisco)'

In this case, the name is “ACE XML Gateway (Cisco)”.

The program is very fast and simple, it detects WAF well. But due to the fact that it is impossible to change the User-Agent, sometimes the program is not able to identify the web protection for the reason that the server rejects the requests of this tool with the default User-Agent.

From our small list of test sites, this program was able to determine the WAF type for two out of four.

identYwaf

identityYwaf is another easy-to-use yet powerful program. Similar to the previous one, this is an identification tool that can recognize the type of web security (eg WAF) based on the server's response. Moreover, this tool makes additional queries to determine which vulnerability categories the WAF is targeting. After completing the scan, the tool gives an estimate (in percentage) of the complexity of exploiting vulnerabilities (if any).

Currently, this program can detect more than 80 different protection products (for example, aeSecure, Airlock, CleanTalk, CrawlProtect, Imunify360, MalCare, ModSecurity, Palo Alto, SiteGuard, UrlScan, Wallarm, WatchGuard, Wordfence, etc.), while the knowledge base is constantly expanding.

identityYwaf performs verification in two ways:

• non-blind when WAFs are identified based on their typical messages. For example, “403 ('<title>Attention Required! | Cloudflare</title>…Sorry, you have been blocked…Why have I been blocked…This website is using a security service to protect itself from online attacks…The action you just performed triggered the security solution…You can email the site owner to let them know you were blocked')”. This message clearly contains the name of the file firewall, but at the same time it may be an attempt to confuse the penetration tester.
• blind when the WAF is identified based on the locks of the payload sent to it

An example of a web application firewall scan:

python3 identYwaf.py admitad.com

Let's examine the output of the program.

Defined the type of web protection by non-blind method, this is Wordfence from Defiant:

[+] non-blind match: 'Wordfence (Defiant)'

A total of 45 tests were performed with different payloads (non-dangerous queries, which are very similar to queries for exploiting vulnerabilities):

[i] running payload tests... (45/45)

The result is presented in the form of a diagram, in which the point is not a blocked payload (no protection against this attack), but a cross is a block from WAF in response to the sent payload):

[=] results: '..............x...x.xx.xxxxx....x.........xx.'

The complexity of exploiting potential attacks is easy:

[=] hardness: easy (26%)

Blocked attack categories:

[=] blocked categories: SQLi, XSS, XXE, PT

Final solution based on blind identification method and its probability:

[+] blind match: 'Wordfence (Defiant)' (100%)

Another test:

python3 identYwaf.py wise.com

Pay attention to the larger number of blocked payloads, categories and, as a result, a higher complexity of exploiting potential vulnerabilities:

[=] results: '....xxx.x...xx.xx.x.xxxxxxxx...xx...x..x.xxx.'
[=] hardness: hard (53%)
[=] blocked categories: SQLi, XSS, XPATHi, XXE, NOSQLi, PHPi, PT

The non-blind method identified the web protection as CloudFlare, the blind method generally came to the same conclusion:

[+] blind match: 'CloudFlare' (98%), 'Incapsula (Incapsula/Imperva)' (82%), 'SecureSphere (Imperva)' (80%), 'Application Security Manager (F5 Networks)' (80%), 'Tencent Cloud|Waterproof Wall (Tencent Cloud Computing)' (78%)

The WAFW00F utility was unable to detect the web protection of ebay.com, let's see if identYwaf can handle:

python3 identYwaf.py ebay.com

This example clearly demonstrates the advantage of two independent WAF identification methods:

The uninformative message “<title>Access Denied</title>…You don't have permission to access” did not allow WAF to be identified by fingerprints.

But scans using a variety of payload types have identified the web protection as Kona Site Defender, manufactured by Akamai Technologies:

[+] blind match: 'Kona Site Defender (Akamai Technologies)' (100%)

We also collected information about the blocking rules, which specific categories of vulnerabilities are filtered:

[=] results: '....x...xx.xxxxx.xxxxxxxxxxxx.xxx......xx.xx.'
[=] hardness: hard (60%)
[=] blocked categories: SQLi, XSS, XPATHi, XXE, PHPi, ACE, PT

If you want to change the User Agent to random, then add the --random-agent option:

python3 identYwaf.py --random-agent ebay.com

If you want the payload to be POSTed, add the --post option:

python3 identYwaf.py --post apple.com

With the --proxy=PROXY option, you can specify the HTTP proxy address (for example, “http://127.0.0.1:8080”). But in my tests it was not possible to use SOCKS proxy (Tor network).

But with the help of the Privoxy program this problem is solved.

Installation Privoxy on Kali Linux, Debian, Linux Mint, Ubuntu and their derivatives:

sudo apt install tor privoxy

Note: configuration in Arch Linux/BlackArch is done in the same way, but with the exception of the command to install packages. Instead of the above, you need to use

sudo pacman -S tor privoxy

Add the lines to the /etc/privoxy/config file:

forward-socks4 / localhost:9050 .
forward-socks4a / localhost:9050 .
forward-socks5t / localhost:9050 .

In the next commands identYwaf uses the Tor network as a proxy:

sudo systemctl start tor
sudo systemctl start privoxy
python3 identYwaf.py --proxy=http://localhost:8118/ www.ebay.com

You can also set the timeout time (waiting for a response) and the time interval between requests (no default), details about these options can be found at https://en.kali.tools/?p=1591.

WhatWaf

WhatWaf is an advanced web application security detection tool that aims to give you an answer to the question, “Does a web server use WAF, and which one?” In the course of its work, WhatWaf detects the presence and type of a web application firewall and tries to find bypasses for the specified firewall on the specified target.

This tool can detect over 70+ different web application firewalls and tries over 30+ different tampering techniques.

WhatWaf program is the most flexible in setting, you can find a complete list of options in the program card: https://en.kali.tools/?p=1603

Usage:

python3 ./whatwaf -u https://SITE.COM

For example:

python3 ./whatwaf -u https://ebay.com

Please note that you can specify the address of a page with parameters as a URL (for example, https://miloserdov.org/?p=6533). Moreover, it is even recommended – the parameters will be used to transfer the payload, which will allow you to more accurately determine how to bypass WAF.

Also note that the URL is preceded by the http or https protocol. The fact is that the program uses HTTP by default, and all modern sites, especially those behind WAF, use HTTPS. Therefore, the protocol must be specified explicitly. There is an option --force-ssl, which, judging by the description, makes the program start using HTTPS by default, but this option did not work in my tests.

The first thing that catches your eye is the long work of this utility. The fact is that WhatWaf is not only trying to define web protection, but also looking for ways to bypass the web application firewall. The bypasses are to use tamper scripts, which modify the payload so that, on the one hand, it (almost) does not lose functionality, but at the same time it ceases to be detected by web protection filters, which usually work based on regular expression searches. Examples of tamper operation: replacing spaces with plus signs, inserting unnecessary comments or meaningless characters, replacing a space with several spaces, URL encoding or double URL encoding, case changes, inserting meaningless tags, etc.

The FIREWALL line contains the detected web application firewall:

[14:15:25][FIREWALL] AkamaiGHost Website Protection (Akamai Global Host)

Consider the result of the following scan:

python3 ./whatwaf -u https://apple.com

In the FIREWALL lines, we see possible web protection systems:

[14:23:45][FIREWALL] Apache Generic
[14:23:45][FIREWALL] Shadow Daemon Opensource (WAF)

And after the line

[14:25:08][SUCCESS] apparent working tampers for target:

it lists the discovered methods of bypassing web application firewall filters.

If you only need to find out the WAF type and want to skip the bypass check, then use the --skip option:

python3 ./whatwaf -u https://apple.com --skip

If you want to specify multiple targets, save them to a file (one URL per line) and run the program with the -l option:

python3 ./whatwaf -l url.txt

You can change USER-AGENT. This can be done with the -ra option (in this case, a random USER-AGENT will be selected):

python3 ./whatwaf -u https://apple.com --skip --ra

Alternatively, with the --pa option, you can specify a specific User Agent:

python3 ./whatwaf -u https://apple.com --skip --pa 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36'

WhatWaf supports working through a proxy, there is a --proxy option for this:

python3 ./whatwaf -u https://apple.com --skip --proxy=socks5://127.0.0.1:9050

There are several options specifically for the Tor network. With the --check-tor option, you can check your Tor connection:

python3 ./whatwaf --check-tor

But in my case, this option did not work correctly, it kept showing the message “[WARN] it appears Tor is not configured properly”, while WhatWaf worked fine through Tor.

There is a special --tor option for anonymous scans:

python3 ./whatwaf -u https://apple.com --skip --tor

The tor service is assumed to be listening on port 9050, if you have a different configuration use the -tP option to specify your own port.

The -W option makes the utility additionally try to determine the version of the web server.

To view the cached results of previous scans, run the utility with the -uC option:

python3 ./whatwaf -uC

To update the program, go to the folder with it and run

python3 ./whatwaf --update

either do

git pull

Online service for detecting Web Application Firewalls

I have collected all the considered tools on the page of one online WAF (Web Application Firewall) detection and identification service: https://w-e-b.site/?act=wafw00f

You just need to enter the domain name of the site you are interested in and the service sequentially scan with each program.

It was even possible to preserve the color output for the two utilities (there are many different escape sequences for coloring the output in the terminal, so converting a color from Bash to HTML is not always a trivial task).

In general, it is convenient to compare the scan results of all three tools at once.