How to discover subdomains without brute-force
Subdomain discovery is an important part of information gathering. More details about subdomains in the article “How to search subdomains and build graphs of network structure with Amass” (although Amass itself was updated to version 3, and examples of commands in that article are given for Amass 2.x – therefore, a new article on Amass 3.x is planned).
SubFinder tutorial
There are many tools for subdomains discovery, and in this article I want to talk about one more – SubFinder, this program can compete with Amass (although no one forbids using them together for the fullest search and full coverage of functions).
Advantages of SubFinder:
- The program is up to date and supports many information sources.
- By default, it works without brute-force and quickly enumerates a large number of subdomains.
- If you wish, you can enable subdomains brute force (dictionary attack).
- It is very easy to use (unlike Amass, which has already changed its options several times, even the names of executable files have changed, and with which it is not even always clear – what kind of search have we started?!)
How to install SubFinder in Kali Linux
The program is written in Go and you need to start with the installation of the Go compiler:
Open the .bashrc file in the user directory with any text editor:
gedit ~/.bashrc
And to create new environment variables, add the following lines to this file:
export GOPATH=/home/git/go export GOROOT=/usr/local/src/go export PATH=${PATH}:$GOROOT/bin:/home/git/go/bin
When you are ready, save your changes and close the file.
These changes will take effect after the reboot. Instead of restarting the computer, run:
source ~/.bashrc
The following command automatically detects and downloads the latest version of the Go language files:
wget `curl -s https://golang.org/dl/ | grep -E -o 'https://[a-z0-9./]{5,}go[0-9.]{3,}linux-amd64.tar.gz' | head -n 1`
Extract the downloaded archive:
tar zxf go*.linux-amd64.tar.gz
Move files to $GOROOT, which we specified in ~/.bashrc.
sudo mv go $GOROOT
Then do:
go get github.com/subfinder/subfinder cp /home/git/go/bin/subfinder /usr/local/src/go/bin/ subfinder -h
API keys for SubFinder
Currently, SubFinder retrieves information from 31 sources, 6 of which only work if you specify API keys. They are:
If you do not want to obtain API keys (although this is free and does not require much effort), then this is not a problem – the program will simply use other services that do not require it.
For example, without installing API keys, the command
subfinder -d sberbank.ru
found 377 subdomains. And with the API keys I had (Virustotal, SecurityTrails and Shodan), this same command found 386 subdomains. That is, there is a difference.
API keys can be written in the configuration file, which is located on the path $HOME/.config/subfinder/config.json. When specifying API keys with options (see below), they are also automatically saved to the specified file.
How to find site subdomains
To passively collect information, it is sufficient to use the -d option followed by the target domain:
subfinder -d SITE.com
For example, find the youtube.com subdomains:
subfinder -d youtube.com
Found 5674 subdomains (not all of them are relevant).
Remove wildcard subdomains
Sometimes the program displays the following message:
Found Wildcard DNS at vk.com
It means that the wildcard character is used in the DNS records of the domain being analyzed, which means that checking any subdomain of the form
ANYTHING.site.com
will always return an IP address.
To exclude random subdomains, use the -nW option, for example:
subfinder -d vk.com -nW
Result:
Resolving 9641 Unique Hosts found Total 952 Unique subdomains found for vk.com
That is, a total of 9641 unique hosts were found, but only 952 unique domains were found:
Similarly, for yandex.ru:
subfinder -d yandex.ru -nW
How to save results in files
The following options are available to save the results:
-o string Name of the output file (optional) -oD string Directory to output results to -oJ Write output in JSON Format -oT Use aquatone style json output format
For example, to search for site subdomains (-d yandex.com), exclude subdomains from wildcard characters (-nW) and save the data to a file (-o yandex.com.sub):
subfinder -d yandex.com -nW -o yandex.com.sub
How to use specific name servers (DNS) to check subdomains
To check the subdomains system settings are used DNS servers. You can specify any other DNS server or several, for this there are options:
-r string Comma-separated list of resolvers to use -rL string Text file containing list of resolvers to use
For example, to use 8.8.8.8 and 1.1.1.1 DNS servers:
subfinder -d kali.org -r 8.8.8.8,1.1.1.1
Recursive Subdomain Search
For a recursive search (when looking for subdomains of already found subdomains), specify the -recursive flag:
subfinder -d vasexperts.ru -recursive
Subdomains of the fourth and higher levels can be found in the usual search via passive source. Probably the most useful this option will be while brute-force.
Tweaking some data sources
The program has two options --set-settings and --set-config, in the description of which it is written that they accept a comma-separated list of settings.
With the first --set-setting option you can fine-tune some data sources. For example, you can specify the number of Censys pages to check, this is done by the following command:
subfinder -d freelancer.com --sources censys --set-settings CensysPages=2 -v
To check all the pages returned from censys, you can use the "all" option.
The following settings are currently supported:
- Censyspages
- Askpages
- BaiduPages
- Bingpages
Specifying the key API in the command line
Using the --set-config option, you can specify API keys directly in the command to be run:
subfinder -d freelancer.com --set-config VirustotalAPIKey=0x41414141,PassivetotalUsername=hacker,PassivetotalKey=supersecret
Remember: after successful use of the API key, it will be added to the configuration file and after that it will be used each time you launch SubFinder
Selecting sources to find information
You can choose the sources that you want to use to collect information, or vice versa, exclude one or more sources:
-exclude-sources string List of sources to exclude from enumeration -sources string Comma separated list of sources to use (default "all")
For example, to search for subdomains of the sberbank.ru site from sources such as Securitytrails, Shodan and Virustotal, run the following command:
subfinder -d sberbank.ru -sources securitytrails,shodan,virustotal
Pay attention to the spelling of the sources – they are written with a small letter, there are no spaces between the words.
How to brutus-force subdomains
With SubFinder can also brute-force subdomains. To use this mode, you must specify the option -b, as well as the option -w after which specify the dictionary. By default, passive mode and brute force will be launched, if you want to use brute force exclusively, specify the -no-passive option.
Create a small dictionary:
echo -e "admin\r\nlogin\r\ndash\r\ndashboard\r\nteam\r\nadministration\r\na\r\nold\r\nback\r\nbackup\r\ntest\r\npanel" > sub.dom
An example of an dictionary attack:
subfinder -d hostland.ru -no-passive -b -w sub.dom
How to search subdomains for a large number of sites
If you have a list of sites, instead of the -d option, use the -dL option, after which you need to specify a file with a list of target domains.
SubFinder Help
All SubFinder options:
-b Use bruteforcing to find subdomains -d string Domain to find subdomains for -dL string List of domains to find subdomains for -exclude-sources string List of sources to exclude from enumeration -nW Remove Wildcard Subdomains from output -no-color Don't Use colors in output (default true) -no-passive Do not perform passive subdomain enumeration -o string Name of the output file (optional) -oD string Directory to output results to -oJ Write output in JSON Format -oT Use aquatone style json output format -r string Comma-separated list of resolvers to use -rL string Text file containing list of resolvers to use -recursive Use recursion to find subdomains -set-config string Comma separated list of configuration details (default "none") -set-settings string Comma separated list of settings (default "none") -silent Show only subdomains in output -sources string Comma separated list of sources to use (default "all") -t int Number of concurrent threads (default 10) -timeout int Timeout for passive discovery services (default 180) -v Verbose output -w string Wordlist for doing subdomain bruteforcing
Online subdomain search
Now there are already two free online subdomain search services:
Related articles:
- Perimeter analysis for website security audit (90.6%)
- The fastest subdomains enumeration and monitoring (also works on Windows!) (63.4%)
- FinalRecon: a simple and fast tool to gather information about web sites, works on Windows (60%)
- TIDoS-Framework: Web Application Information Gathering and Manual Scanning Platform (53.8%)
- How to search subdomains and build graphs of network structure with Amass (53.3%)
- Guide to GPS Metadata in Photos (Part 2): How to understand, extract and convert geographic coordinates (RANDOM - 24.1%)
I would be cool if you will review Spyse.py. We made this wrapper for free, based on findsubdomains.com.
Hello! Do you mean https://github.com/zeropwn/spyse.py?
Yeah that's awesome tool. i spoke with Dominik(zeropwn), he is goint to update his wrapper.