How to search subdomains and build graphs of network structure with Amass
What are subdomains
Subdomains are domains of the third, fourth and subsequent levels. Take as an example the site kali.tools. Its subdomain (third level domain) is en.kali.tools. General view of the third level subdomains of this site *.kali.tools. Domains can be the fourth test.en.kali.tools, exp.en.kali.tools and subsequent levels.
Subdomains and subfolders are not the same thing. An example subfolder for kali.tools is kali.tools/all. Both subdomains and subfolders can contain different web application, different content management systems are installed, after all, there may be two completely different sites.
The difference between a subdomain and a subfolder is that for a subdomain a DNS name server may contain a record. That is, for each domain, the DNS server has a record in which the domain name (kali.tools) is mapped to an IP address. So, the DNS server can have such records for subdomains. And IP addresses for subdomains can be specified different. This means, for example, that kali.tools can be placed on one server, en.kali.tools - placed on another, and test.en.kali.tools again on another server and even in another country.
The subdomain discovering is an important part of the initial stage of pentesting when information about the target site/host/network is gathered. Since knowledge of other subdomains allows you to expand the perimeter for the attack, or to get an idea about the structure of the target's network.
How to search for subdomains
Subdomain discover techniques are conventionally divided into passive and active.
In a passive search, data is requested on services that collect this information — there are quite a lot of such services. Data can also be collected from information about issued certificates, revealed via search engines, reverse DNS (a researcher gets the all hosts on the target IP and a researcher filters out subdomains), etc.
The active technique is brute-force. It can be brute-force via HTTP requests: if the status code is 200 (for en.kali.tools), then there is such a subdomain, and if the status code is 404 (for ch.kali.tools) it means there is no such host. In fact - such brute force is used infrequently. It is much more convenient (and faster) to query DNS servers and, based on their responses, determine whether a subdomain exists. But the first option is sometimes the only possible way if the subdomains are located on the same IP address, and the wildcard character is specified in the DNS records.
Amass is a powerful subdomain discover program
The main purpose of Amass is to search for subdomains. Also the program can build graphs, for example:
And further:
Maltego Import Wizard:
Graph in Maltego:
These graphs are examples from developers - but by the end of this article we will build our own graphs.
The feature of Amass (there are a lot of similar tools and online services) is that it has collected a large number of different techniques and uses many services to obtain data, and also knows how to brute-force subdomains.
The program is cross-platform - it works also on Windows, Linux.
The program has many options and you can quite accurately configure the methods of domain discovering.
How to install Amass in Linux
Amass can be installed in almost any Linux distribution in the manner described below - I checked it on Kali Linux, it definitely works there.
sudo apt install golang go get -u github.com/OWASP/Amass/... cd ~/go/src/github.com/OWASP/Amass/ go install ./... sudo mv ~/go/bin/amass* /usr/bin/ sudo mkdir -p /usr/share/amass/wordlists/ sudo mv ~/go/src/github.com/OWASP/Amass/wordlists/* /usr/share/amass/wordlists/
How to install Amass in Windows
For Windows, there are also compiled binary files. Go to the releases page. Binaries are available not for all versions of Windows, find the latest version for Windows:
Download and unzip the archive.
Open a command prompt, to do this, click Win+x, there select Windows PowerShell. To go to the command line, run:
cmd
Using the cd command, go to the folder with the program files. For example, I have a folder C:\Users\Alex\Downloads\amass\, then the command is as follows:
cd C:\Users\Alex\Downloads\amass\
To make sure that the program works, show help:
amass -h
Note that programs can be run as amass (that is, without the .exe extension), or as amass.exe (that is, with the extension).
An example of starting a scan:
amass -d kali.org -passive
Everything else that is said in this manual about Amass applies equally to Windows and Linux versions.
How to search subdomains with Amass
The program has one mandatory option -d, followed by the domain for which subdomains will be searched for.
Initial usage:
amass -d kali.org
Launched in this manner, the program will run for a long time, because by default the program:
- uses passive subdomain acquisition techniques (this is a quick step)
- performs dictionary brute-force attack to search for subdomains (long)
- for found subdomains, alters their names (for example, adds numbers) and checks these names also
- for the found subdomains, it again searches for even deeper subdomains (of the fourth, fifth, and so on) levels — that is, performs recursive brute force
To further expand the scope of the search, pay attention to the options:
- -active, it means enable active intelligence methods: an attempt to collect zone transfers and names from certificates
- -whois it means that the search will also use reverse whois information, which may contain domains
If you are interested in seeing which IP addresses have the hosts found, use the -ip option.
The -v option causes a source of information to be displayed for each name found, and at the very end of the program, a brief summary is displayed.
Passive subdomain search
Amass has the -passive option; when it is used, brute-force is not performed - due to this, the search for subdomains is greatly accelerated - only information from other online sources is collected:
amass -d kali.org -passive
You can add -v and -whois:
amass -d kali.org -passive -v -whois
Results:
How to speed up Amass
In addition to the option -passive option, which completely disables the search for names via brute-force, there are options:
- -noalts it disables the generation of new names, performed by changing the already discovered subdomains
- -norecursive - disable recursive enumeration of domains of higher levels
Dictionaries to brute-force subdomains
Several dictionaries are shipped with Amass, they are located in the /usr/share/amass/wordlists/ directory:
- all.txt
- asnlist.txt
- bitquark_subdomains_top100K.txt
- deepmagic.com_top500prefixes.txt
- deepmagic.com_top50kprefixes.txt
- fierce_hostlist.txt
- jhaddix_all.txt
- namelist.txt
- nameservers.txt
- sorted_knock_dnsrecon_fierce_recon-ng.txt
- subdomains.lst
- subdomains-top1mil-110000.txt
- subdomains-top1mil-20000.txt
- subdomains-top1mil-5000.txt
- user_agents.txt
Amass uses one of them, or a dictionary is already built into it. You can specify your own dictionary with the -w option.
Making graphs with Amass
You need to start with a search with any options, but in addition you need to specify the option -do after which specify the path to the file where the results will be saved. You can also use the -oA prefix option, which will create several files in different formats, including a file of the form *_data.json - which we need to compile graphs.
Note that the -d option can be used several times to specify multiple root domains. Even with a single -d option, you can specify multiple domains separated by commas. Or you can use the option -blf after which specify the path to the file with the list of domains.
An example of launching a scan at once for the two domains kali.org and offensive-security.com with saving the obtained data in data.json:
amass -d kali.org -d offensive-security.com -v -whois -do data.json
An example of a quick scan command with saving data in all formats (so as not to waste time on brute-force with the -w option, a dictionary in several lines 1.txt is specified):
amass -d kali.org -d offensive-security.com -v -ip -w 1.txt -noalts -norecursive -oA kali-offsec
The results are obtained and saved to the kali-offsec_data.json file. Using the program amass.viz, you can convert the data into graphs of various formats. After the -i option, you need to specify the file with the original data. You can convert to several or all formats at the same time:
amass.viz -i kali-offsec_data.json --maltego of1.csv --visjs of2.htm --gexf of3.gexf --d3 of4.htm --graphistry of5.json
Files with the .htm extension can be opened directly in the browser - the circles are moving:
The second graph is also interactive:
To open the file with the .csv extension (I have it called of1.csv), run Maltego, there choose the creation of a new graph:
Click the Import | Export and select Import Graph from table:
There will be a small tutorial - scroll through it. Select a file:
Additional settings will be available - you can leave the default.
I got this:
Conclusion
Some options and even commands from the package are not covered in these instructions for using Amass. Full details and a list of Amass options can be found here: https://github.com/OWASP/Amass
Amass is powerful enough and easy to use. Although I do not fully understand the logic of its work. In previous versions, brute force was disabled by default. In the latest version, except with the use of -passive option, it is impossible to disable brute force. However, due to the -passive option, some other functions do not work, such as saving data to a file. At the same time, specifying a dictionary with -w, it seems, does not change anything - a long brute-force is performed for some large dictionary. It feels that with the -passive option, the program finds more results than without it (although this is completely strange).
When I first started the latest version, I just thought that the program was not working, because the first 5-10 minutes it outputs nothing. If you launch Wireshark, you can see that the program continuously sends DNS requests - that is, it works.
Despite the current shortcomings (I hope they will be corrected), Amass is one of the most (or even the most) best program for searching subdomains.
Search subdomains online
I added online access to Amass on SuIP.biz, so if you can search for subdomains even without installing the program: https://suip.biz/?act=amass
Related articles:
- FinalRecon: a simple and fast tool to gather information about web sites, works on Windows (77.5%)
- Perimeter analysis for website security audit (75.1%)
- How to find out hostnames for many IP addresses (74.5%)
- TIDoS-Framework: Web Application Information Gathering and Manual Scanning Platform (72.1%)
- The fastest subdomains enumeration and monitoring (also works on Windows!) (70%)
- Active Directory comprehensive guide, from installation and configuration to security auditing. Part 1: Introduction to Active Directory (concepts, usage, difference from Workgroup) (RANDOM - 0.8%)