Utilities for information gathering, OSINT and network analysis in Windows and Linux
Table of contents
1. How to install Habu on Windows
2. Program to find sites with the same IP in Windows
3. Windows program for searching by username
5. How to scan ports in Windows
6. How to scan a file by all antivirus programs in Windows
8. Quick way to find website domains running in the IP range
9. How to validate bank card number. Recovering missing digits in a card number
10. How to find site subdomains in Windows
11. How to find out IP of a web-site in Windows
12. How to know the domain by IP in Windows
13. Retrieving Host Information from Shodan
14. Subdomains Search in Windows
15. ARP scanning and spoofing in Windows
16. Base64 Encoding and Decoding on Windows
17. How to extract domains and IP addresses
18. How to check whether antivirus works
19. How to find traffic forward devices
20. Calculating Hashes on Windows
21. How to find out my IP and IPv6 on Windows
22. Network Interfaces in Windows
23. How to view HTTP site response headers in Windows
24. Fernet Encryption on Windows
25. XOR string operation on Windows
How to install Habu on Windows
In a previous article, I showed how to install Python on Windows. I mentioned that thanks to Python, you can run programs and scripts that in my tutorials I run in Kali Linux or in another Linux distribution. In this article, I will demonstrate how you can use Python on Windows by running utilities that can help you scan devices on your local network, perform an initial information gathering about web applications and find users on popular sites, and even perform some simple attacks.
We will do all this with Habu – a set of various tools.
Habu programs are written in Python, so they can work on both Linux and Windows. Moreover, these utilities are of particular interest specifically for Windows users, since there are no alternatives for this operating system. For this reason, I will show the operation of these programs on Windows.
You need to start by installing Python and PIP on Windows. Then run the command:
pip install habu validators matplotlib tldextract prompt-toolkit
As you can see, many dependencies have been installed.
Some functions (for example, for tracing) require the installed Winpcap or Npcap program. You can read more about these programs and why they are needed at the beginning of the article “Hacking Wi-Fi without clients on Windows (using Wireshark and Npcap to capture PMKID)”.
You can install Npcap or an older version of Winpcap, but another option is to install Wireshark on Windows, which comes with Npcap. If you are interested in questions of network research, penetration testing, then Wireshark is useful for you, install it.
To update Habu and all the dependencies, just run the installation command again – if new versions of packages are found, they will be updated:
pip install habu validators matplotlib tldextract prompt-toolkit
Now let's get acquainted with the Habu utilities – there are a lot of them, more than 70, a complete list of them, as well as options, you will find on the page https://github.com/fportantier/habu. In this article, I have picked up the most interesting of them.
Program to find sites with the same IP in Windows
On one server there can be several websites at once. To find sites on one server, that is, having one IP address, use a command of the form:
habu.vhosts DOMAIN
For example, to find sites that run on the same server as the suip.biz site, you need to run the command:
habu.vhosts suip.biz
To prevent the program from using the cache, add the -c option:
habu.vhosts suip.biz -c
As you can see from the screenshot, the program determines the IP address of the entered domain and then using the search engine Bing finds sites with the same IP address.
Instead of a domain, you can specify an IP address:
habu.vhosts 185.117.153.79
Windows program for searching by username
Using the following utility, you can find user profiles on social media and on popular sites. To start the search you need to use a command of the form:
habu.usercheck USERNAME
For example, to search for profiles with the username Mi-Al:
habu.usercheck Mi-Al
Search for profiles with the login miloserdov:
habu.usercheck miloserdov
Traceroute in Windows
Using traceroute, you can find the IP addresses of network nodes (hopes) through which the packet passes to the specified host. To do this, use a command of the form:
habu.traceroute HOST_OR_IP
For example, I’m interested in what network route from my computer to suip.biz:
habu.traceroute suip.biz
See more information about traceroute in the articles:
How to scan ports in Windows
One of the best port scanning software is Nmap – it supports several scanning methods and has many additional features. You can use it, but if you just need to check if a certain port is open, then use the habu.tcpscan utility for this.
To scan ports, run a command of the form:
habu.tcpscan -p PORT1,PORT2 DOMAIN_OR_IP
As ports, you can specify one port, several ports separated by commas, or a range of ports through a hyphen, for example:
habu.tcpscan -p 80,8000-8100 suip.biz
Additionally, options may come in handy:
-f TEXT Flags to use (default: S) -s TEXT Time between probes (default: send all together) -t INTEGER Timeout for each probe (default: 2 seconds) -a Show all responses (default: Only containing SYN flag) -v Verbose output
How to scan a file by all antivirus programs in Windows
Using the Virustotal service, you can scan a single file at once with many antiviruses. You can do this check directly on the Windows command line using the habu.virustotal utility.
The habu.virustotal program works as follows: for the specified file, it calculates the hash and, without sending the file, makes a request whether Virustotal has a report for the file with such a checksum. If there is a ready report, then the program receives it. If there is no report, the program sends a suspicious file to Virustotal and expects to receive a report.
To use habu.virustotal, you need to get the Virustotal API key – this is a free key, it is enough to get it once and after that you can use it without restrictions.
- To get the Virustotal API key, open the page https://www.virustotal.com/gui/join-us and register (including verification of the email address).
- Log in to your account and go to https://www.virustotal.com/gui/user/YourUsername/apikey
- Find the ‘API key’ section.
- Save your API key.
To use habu.virustotal, you must first save the received key to the configuration, this is done by a command of the form:
habu.config.set VIRUSTOTAL_APIKEY YOUR_KEY
To check, do:
habu.config.show
Instead of a key there will be asterisks – this is normal.
Now you can check any file on the command line by running the utility as follows:
habu.virustotal PATH\TO\FILE
For example, to check the Documents\test1.pptx file with verbose output (-v):
habu.virustotal Documents\test1.pptx -v
Output Example:
{ "md5": "11d9310ba2f59639c1b252b12466ce40", "permalink": "https://www.virustotal.com/file/2a95f1c2aa3a0629054bef973787d144156a38659cc2a1be56eb8f764b387af5/analysis/1575116510/", "positives": 0, "resource": "2a95f1c2aa3a0629054bef973787d144156a38659cc2a1be56eb8f764b387af5", "response_code": 1, "scan_date": "2019-11-30 12:21:50", "scan_id": "2a95f1c2aa3a0629054bef973787d144156a38659cc2a1be56eb8f764b387af5-1575116510", "scans": { "ALYac": { "detected": false, "result": null, "update": "20191130", "version": "1.1.1.5" }, "AVG": { "detected": false, "result": null, "update": "20191130", "version": "18.4.3895.0" }, "Ad-Aware": { "detected": false, "result": null, "update": "20191130", "version": "3.0.5.370" }, "AegisLab": { "detected": false, "result": null, "update": "20191130", "version": "4.2" }, "AhnLab-V3": { "detected": false, "result": null, "update": "20191130", "version": "3.16.5.25880" }, "Alibaba": { "detected": false, "result": null, "update": "20190527", "version": "0.3.0.5" }, …………………………………. …………………………………. …………………………………. …………………………………. ………………………………….
In the information received, first of all, pay attention to the line:
"positives": 0,
It shows how many antiviruses consider that the sent file is a malicious program.
Next is the information on the results of scanning by each antivirus.
WHOIS in Windows
WHOIS is a request and response protocol for retrieving information from a database. Most often, WHOIS is used to obtain registration information about owners of domain names, IP addresses and autonomous systems.
On Linux, whois can be used directly from the command line. But on Windows, such a utility is missing.
Using commands of the form:
habu.whois.domain DOMAIN habu.whois.ip IP_ADDRESS
You can get WHOIS records for the domain and IP address, respectively.
For example:
habu.whois.domain kali.tools
Quick way to find website domains running in the IP range
If a web server is found on the remote computer to which ones connect by IP address, then it is sometimes difficult to determine which domain this site has?
Currently, most websites have SSL certificates, that is, they support data transfer via HTTPS protocol. SSL certificates contain the name of the domain for which this certificate is issued. Therefore, when you find a web server and site, the easiest way to find out a domain name is to connect to port 443 and extract information from the certificate.
This is exactly the kind of technique habu.cert.names uses. This program can work with ranges of IP addresses and very quickly get domain names of sites that runs on each web server.
To start a scan, use the command of the form:
habu.cert.names NETWORK
Example:
habu.cert.names 185.117.153.0/24
How to validate bank card number. Recovering missing digits in a card number
Bank card numbers, as well as some other numbers, use a check digit calculated using the Luhn algorithm.
The most common uses for calculating a check digit:
- Numbers of all bank cards
- Numbers of some discount cards
- Social Security Codes
- IMEI codes.
- Calculation of the control mark of a single 8-digit number of a railway carriage on Russian Railways.
- ICCID calculation (integrated circuit card identifier) is a unique serial number of a SIM card.
With this check digit, you can determine if the number is correct.
For example, card number 4276100014796930, is this the correct number or not?
To check, you need to run habu.crack.luhn as follows:
habu.crack.luhn 4276100014796930
If nothing is displayed, then the number is incorrect.
Now check the card number 4276100013796930:
habu.crack.luhn 4276100013796930
The program in response showed us this number, which means that such a card number is valid.
Consider an example where one or more digits in a card number are unknown. In this case, we write the x symbol instead of the missing numbers, for example:
habu.crack.luhn 4276-1000-13xx-6930
In this case, all options of possible (valid) numbers will be displayed:
How to find site subdomains in Windows
Subdomains are domains of the third and subsequent levels. For example, for the suip.biz website, the subdomains are www.suip.biz, test.suip.biz, en.suip.biz, etc.
There are different ways to search for subdomains – both brute force, and using various services that have accumulated statistics. One of the fastest and most effective ways is to analyze Certificate Transparency logs. Searching for subdomains is quick, but it takes some time to verify them (whether they have IP addresses).
To search for subdomains, run a command of the form:
habu.crtsh DOMAIN
For example, to search for subdomains of ebay.com:
habu.crtsh ebay.com
Some useful options that you can use with this utility:
-c Disable cache -n Disable DNS subdomain validation -v Verbose output
How to find out IP of a web-site in Windows
You no longer need to use third-party services and sites if you want to find out the IP address of any domain.
To determine the IP and IPv6 of any site, run a command of the form:
habu.dns.lookup.forward DOMAIN
For example, to find the IP address of the site w-e-b.site:
habu.dns.lookup.forward w-e-b.site
How to know the domain by IP in Windows
Speaking about the habu.cert.names utility, I already mentioned that by IP it is not always easy to determine the sites running on the server without using external services. The habu.cert.names utility uses a tricky technique by extracting the domain name from the certificate. Using the habu.dns.lookup.reverse utility, you can also quickly find out a domain by IP. To do this, run a command of the form:
habu.dns.lookup.reverse IP_ADDRESS
For example:
habu.dns.lookup.reverse 157.245.118.66
Retrieving Host Information from Shodan
Shodan service scans IP addresses, open ports and gathers service banners. Using this service, you can find out information about running processes on hosts accessible from the Internet.
As in the case of habu.virustotal, for Shodan you must first obtain the API key. It is enough to register – a free key is enough for us. Go to the site https://account.shodan.io/, register and find your API key.
Now you need to add the API to the Habu configuration with a command of the form:
habu.config.set SHODAN_APIKEY YOUR_KEY
To check, do:
habu.config.show
Instead of a key there will be asterisks – it is alright.
For requests to Shodan, there are two utilities – one is designed to obtain information about domains and runs as follows:
habu.shodan DOMAIN
The second will show the available information, including about the running services and the grabbed banners about the IP address, it starts like this:
habu.shodan.query IP_ADDRESS
Example of requesting information to Shodan about the suip.biz domain:
habu.shodan suip.biz
An example of obtaining information about IP from Shodan:
habu.shodan.query 185.117.153.79
Subdomains Search in Windows
Again we return to subdomains and virtual hosts. The habu.fqdn.finder module immediately uses many techniques to collect perimeter information:
1. Checks Certificate Transparency Logs
2. Connects to the specified ports, receives SSL certificates and receives FQDN from them
3. Tries DNS Zone Transfer
This module also requires the Shodan API key.
You can run the program without options, but then it will work for a long time without showing anything – it may seem that it is frozen. To display the current tasks in progress, use the -v option. Other options:
-t FLOAT Time to wait for each connection -v Verbose output --debug Debug output --connect / --no-connect Get from known FQDNs open ports SSL certificates --brute / --no-brute Run DNS brute force against domains --links / --no-links Extract FQDNs from web site links --xfr / --no-xfr Try to do a DNS zone transfer against domains --ctlog / --no-ctlog Try to get FQDNs from Certificate Transparency Logs --json Print the output in JSON format
An example of launching an ebay.com site perimeter research with detailed information output:
habu.fqdn.finder ebay.com -v
ARP scanning and spoofing in Windows
ARP is a LAN protocol that allows devices to recognize each other's IP addresses.
If you run ARP sniffing, you can see which devices are present on the local network and what IP and MAC addresses they have:
habu.arp.sniff
Using the habu.arp.poison module, you can perform an ARP spoofing attack, which is the first stage of a man-in-the-middle attack.
Base64 Encoding and Decoding on Windows
Base64 encoding allows you to represent any data – text or binary files as a sequence of printed characters. That is, as a result, binary data that cannot be printed or included in a document or email due to the presence of special characters can be transmitted in the form of an ordinary printed line.
Base64 is used quite widely - for example, quite often files attached to email are encoded in this way and are included in the sent message as a line which is surrounded by special tags. When receiving such email, mail programs see tags, decode the file and show it to you as attachment.
Using the habu.b64 utility, you can encode strings and files in Base64, as well as decode strings and files from Base64.
To encode strings, use a construction of the form:
echo LINE_TO_ENCODE | habu.b64
To encode files, use a command of the form:
habu.b64 FILE_TO_ENCODE
For example, to encode the test.txt file in Base64:
habu.b64 test.txt
As a result, I got the line:
0J/QvtC30LTRgNCw0LLQu9GP0Y4gLSDQstGLINGA0LDRgdGI0LjRhNGA0L7QstCw0LvQuCDRgdC+0LTQtdGA0LbQuNC80L7QtSDRjdGC0L7Qs9C+INGE0LDQudC70LAh
To decode, add the -d option to the command, for example:
echo 0J/QvtC30LTRgNCw0LLQu9GP0Y4gLSDQstGLINGA0LDRgdGI0LjRhNGA0L7QstCw0LvQuCDRgdC+0LTQtdGA0LbQuNC80L7QtSDRjdGC0L7Qs9C+INGE0LDQudC70LAh | habu.b64 -d > file.txt
The decoded data will be output directly to the terminal. The decoded data may contain characters that the terminal does not support, or it may be binary, so I use the > FILE_NAME construct to redirect the output (> file.txt) to the file, that is, the decoded data will be saved in the file.
How to extract domains and IP addresses
If you need to extract domain names or IP addresses from a large number of logs, documents, reports, source code or any text files, then these utilities will help you
- habu.data.extract.domain
- habu.data.extract.ipv4
Using habu.data.extract.domain as an example, let's see how to get a list of domains. The curl command downloads the source code of the page. That is, in the following command, first we get the source code of the web page https://mi-al.ru/waytopay/, and then it extracts all the domains found in the code:
curl -s https://mi-al.ru/waytopay/ | habu.data.extract.domain
How to check whether antivirus works
The habu.eicar module does a very simple thing – it displays a line that, if saved to a file, antiviruses will begin to consider this file a virus.
EICAR (or EICAR-Test-File – from the European Institute for Computer Antivirus Research) is a standard file used to check if the antivirus is working. Essentially not a virus; running as a DOS COM file, it just displays a text message and returns DOS control. The program works in environments that support the execution of 16-bit DOS software, such as MS-DOS, OS/2, Windows 9x, and 32-bit Windows NT. Under 64-bit versions of Windows, the file does not start.
Of course, EICAR does not check how quickly developers respond to viruses and how well infected files are cured – for this you need a “zoo” of fresh viruses. Its task is different: to demonstrate the operability of the antivirus system and indicate which objects are scanned by the antivirus and which are not. For example:
- There is a suspicion that the computer is infected. Is there a resident monitor, or was the virus able to disable it?
- A regular mail worm like VBS.LoveLetter must go through several stages for infection: come to the mail server via SMTP; boot to the computer using the POP3 protocol register in the mail client database; at the command of the user to unpack into a temporary file and start. At what stage will he be seen?
- There are many ways to “drag” a malicious program past the “eyes” of an antivirus: encode in Base64, embed Microsoft Word in an OLE object, in RAR, JPEG, compress it with a packer like UPX. Which of these will the antivirus unpack?
- In addition, antiviruses are not only local, but also network – checking network traffic; with a configuration error, they will either load the server with unnecessary work, or, conversely, skip malicious files.
- Just to see the antivirus response.
- In order to check what the reaction of the antivirus will be, of course, you can use a ‘live’ virus – but it is not always safe and excessively. For this, a standardized file was proposed that does not carry a malicious load.
Usage example:
habu.eicar > virus.exe
How to find traffic forward devices
The following command at first glance may seem to be of little use – as an argument, they take a network range and show the IP address of the router. For example:
habu.gateway.find 192.168.1.0/24
Output Example:
192.168.1.1 14:9d:09:d0:04:7e Huawei Technologies Co.,Ltd
But in fact, everything is a little more complicated: the program does not just look for a router, but checks all devices on the local network for the possibility of redirecting traffic.
For example, on another computer that runs on Linux, I enable traffic forwarding:
sudo sysctl -w net.ipv4.ip_forward=1
And again I start scanning on a Windows computer:
habu.gateway.find 192.168.1.0/24
Now I get two results at once:
192.168.1.4 c0:b6:f9:da:af:3e Intel Corporate 192.168.1.1 14:9d:09:d0:04:7e Huawei Technologies Co.,Ltd
Enabling traffic forward may be the start of a man-in-the-middle attack. That is, if a device is detected with such a function, in addition to the router, you should review it more carefully.
Or, on the contrary, you can search for such devices with the traffic redirect turned on for exploitation, for example, for use as a proxy.
Calculating Hashes on Windows
Using habu.hasher, you can find checksums of both the line and the file.
habu.hasher FILE
Example:
habu.hasher test.txt
How to find out my IP and IPv6 on Windows
To find out your external IP address in Windows, you no longer need to go to third-party services, just run the command:
habu.ip
If you want to know your external IPv6, then run the following command:
habu.ip -6
To see your local IP addresses, run the following command:
habu.ip.internal
Network Interfaces in Windows
To find out the names of network interfaces, their MAC and IP addresses, run the command:
habu.interfaces
How to view HTTP site response headers in Windows
Using the habu.http.headers utility, you can see which headers the target website uses and which cookies it sets. For example:
habu.http.headers https://yandex.ru
Fernet Encryption on Windows
Create your Fernet key with the command:
habu.fernet.genkey
Output Example:
oThsmgmxlGslnelGXMPPcFrpk4Jyf4rSGNdAO8mABrc=
Now save it in the configuration:
habu.config.set FERNET_KEY oThsmgmxlGslnelGXMPPcFrpk4Jyf4rSGNdAO8mABrc=
To check, do:
habu.config.show
To encrypt, use the construct:
echo "Хочу зашифровать эту строку" | habu.fernet
Output Example:
gAAAAABd44yL6_y2nT2BGkcmszkHBmN-uj3t9nqBBLzlNYL3QGBWLMXHGQx3dYmNZRensu0bG6OCsBeQX4Gxc1TuAAesQKFQn3zmHEwwccQ28UZV3XoqGvHNAJQqJZOYsrlFZNV-XTGN
If you have not set the key value in the configuration, then specify it using the -k option:
echo "Хочу зашифровать эту строку" | habu.fernet -k oThsmgmxlGslnelGXMPPcFrpk4Jyf4rSGNdAO8mABrc=
You can also encrypt files and set the verification lifetime, for this use the options:
--ttl INTEGER Time To Live for timestamp verification -i FILENAME Input file (default: stdin) -o FILENAME Output file (default: stdout)
To decrypt, use the -d option:
echo "gAAAAABd44yL6_y2nT2BGkcmszkHBmN-uj3t9nqBBLzlNYL3QGBWLMXHGQx3dYmNZRensu0bG6OCsBeQX4Gxc1TuAAesQKFQn3zmHEwwccQ28UZV3XoqGvHNAJQqJZOYsrlFZNV-XTGN" | habu.fernet -d
Although, with the decoding of non-Latin characters, I had an error.
XOR string operation on Windows
Using the XOR operation of two lines, you can encrypt the text with a password. This encryption is symmetric and weak. XOR encryption can be used in non-critical cases as the easiest way to hide the original string (for example, from automatic scanners).
If you take the lines and perform the XOR operation between them, the result is a meaningless set of characters. Then, if between this meaningless line and any of the original lines, the XOR operation is performed again, then the initial line is obtained again.
The simplest symmetric encryption is based on this: the source text is encrypted with a password using XOR. That is, the XOR operation is performed with the first character of the text and the first character of the password, then the XOR operation is performed with the second character of the encrypted text and the second character of the password, and so on, until the encrypted text ends. Since the password is usually shorter than the encrypted text, when it ends, the transition to the first character of the password is performed again and so on many times.
The result is a meaningless set of characters that can be decrypted with the same password by performing the same XOR operation.
For encryption, use any password that you need to specify with the -k option. Since the resulting string is very likely to have unprintable characters, it is better not to output it to the terminal, but save it to a file - this is done with the -o option or by redirection:
For example, the following command will encrypt the string my secret string with the password mypassword and save it to the crypted.bin file.
echo "my secret string" | habu.xor -k mypassword > crypted.bin
The following command will encrypt the test.txt file and save its encrypted version in test.bin:
habu.xor -k mypassword -i test.txt -o test.bin
For decryption, it is enough to perform the same operation – specify the original password, and the encrypted file as input:
habu.xor -k mypassword -i test.bin
And:
habu.xor -k mypassword -i crypted.bin
In the screenshot above, you see that the strange symbols are displayed in the terminal – various binary data that is interpreted as unprintable characters. A little earlier, we got acquainted with the habu.b64 utility, which allows us to represent binary data as a string of printed characters – we can use it!
For example, I want to encrypt the test.txt file and transfer it not as a binary file, but as a string:
habu.xor -k mypassword -i test.txt | habu.b64
As a result, I got a string of printable characters:
veag36PEp9uj5L3JoNOjyKbgo+pNVFCxwaL8T6Pkvcmh4KL7p9ej4Lz5oN+jwaffot+9wVCw8qPJv8a02KjwscWjz7/OtNOpxUGi/qbtotq9yqDfU6Lzv8K01KnLscNS
To decrypt it, it is enough to perform the inverse conversion – from base64 encoding to binary data, and then decrypt using the same password with which it was encrypted:
echo "veag36PEp9uj5L3JoNOjyKbgo+pNVFCxwaL8T6Pkvcmh4KL7p9ej4Lz5oN+jwaffot+9wVCw8qPJv8a02KjwscWjz7/OtNOpxUGi/qbtotq9yqDfU6Lzv8K01KnLscNS" | habu.b64 -d | habu.xor -k mypassword
Other Habu commands
We examined only some of the teams from the Habu package. You can find the full list at https://github.com/fportantier/habu. In addition to those already considered, I would especially like to highlight the following:
- habu.cymon.ip
- habu.data.extract.domain
- habu.data.extract.email
- habu.data.extract.fqdn
- habu.expand
- habu.host
- habu.nmap.open
- habu.ping
- habu.protoscan
- habu.web.report
- habu.web.screenshot
Some of them are quite specific, some do not quite work properly on Windows at the time of writing – but in any case, it is worth taking a closer look at them when working on Linux, or they may work properly in future Windows updates.
Troubleshooting
[TO BE CONTINUED]
Related articles:
- Revealing the perimeter (CASE) (63.2%)
- How to install ꓘamerka (kamerka) (57.3%)
- Perimeter analysis for website security audit (57.3%)
- FinalRecon: a simple and fast tool to gather information about web sites, works on Windows (56.8%)
- Best Kali Linux tools in WSL (Windows Subsystem for Linux) (Part 1) (56.3%)
- Linux directory structure. Important Linux files (RANDOM - 0.2%)