How to find out what the site works on

To get an idea of how many technologies can be involved in a single page of a website, look at this list.

How websites work

At the very core there are the servers on which the operating system is installed. A web server is installed in this operating system, and various extensions for running various scripts are enabled for this web server (for example, PHP, Python, PERL, Ruby). Other extensions are also enabled for the web server: for security (HTTPS, for example), for faster downloads, data caching, etc.

One or another content management system (CMS), some of the popular engines of the forum, wiki, blog, photo gallery, online store, etc. can be installed on the web server as the software basis of a web site.

This software basis of a site can have its own extensions: SEO plugins, text editors, caching plugins and more.

The site’s software platform can be written using JavaScript frameworks, JavaScript libraries, graphical JavaScript extensions, and more.

A variety of third-party services can already be connected to the operating site: analytics, metrics, commenting systems, CDN, captcha, payment systems, ad networks, cryptominers, fonts, maps, video players.

In general, a lot of technologies can be involved, and to immediately determine everything simply by looking into the source code is not always possible.

To solve this problem – instantly identifying all the technologies involved on the site, a special extension for web browsers named wappalyzer was created.

Wappalyzer is a browser plugin that reveals the technologies used on websites. It uncovers content management systems, eCommerce platforms, web servers, JavaScript frameworks, analytics tools, and much more. All technologies are listed on the already given link.

You can install Wappalyzer for Google Chrome from the link: https://chrome.google.com/webstore/detail/wappalyzer/gppongmhjkpfnbhagpmjfkannfbllamg

You can install Wappalyzer for Firefox here: https://addons.mozilla.org/en/firefox/addon/wappalyzer/

After installing Wappalyzer, a new one will appear among the extension icons. Just go to the site that interests you and click on the Wappalyzer icon.

Example output for HackWare.ru:

Example output for SuIP.biz:

Pay attention to the plugin settings:

It is possible that you will want to disable some of them:

webanalyze is a command line port of wappalyzer

The extension for the web browser is very convenient for ordinary users, but such an implementation does not allow you to check numerous sites in a short time (what if you need to check 1000 or more web sites?), In addition, some of us are just more familiar with working on the command line.

For such cases the webanalyze tool was created, which is essentially a Wappalyzer program, ported to the Go programming language.

How to install webanalyze

Start with Go installation (choose manual installation).

Then run the commands:

go get -u github.com/rverton/webanalyze
webanalyze -update
webanalyze -h

How to use webanalyze

To collect information about the technologies used by the site https://hackware.ru run the command:

webanalyze -host https://hackware.ru

To find out information about the technologies of the site https://suip.biz, as well as information about technologies with 5 links found on the specified page (-crawl 5):

webanalyze -host https://suip.biz -crawl 5

If you need to check many sites, then save them to a file in the format: one site per line. When starting up, specify this file to the program using the -hosts option:

webanalyze -hosts sites.txt

By default, the results are output to standard output in a human-readable format, but using the -output option you can specify the csv format (data is separated by commas, this file can be opened by the program for working with tables like LibreOffice Calc), or you can specify json for output in JSON format. For example:

webanalyze -host https://paysend.pro -output csv

But this command will also output the result to standard output, to redirect the output to a file use the > symbol, for example:

webanalyze -host https://paysend.pro -output csv > paysend.csv

Other options of the webanalyze program:

  -apps string
    	app definition file. (default "apps.json")
  -crawl int
    	links to follow from the root page (default 0)
  -host string
    	single host to test
  -hosts string
    	filename with hosts, one host per line.
  -output string
    	output format (stdout|csv|json) (default "stdout")
  -search
    	searches all urls with same base domain (i.e. example.com and sub.example.com) (default true)
  -update
    	update apps file
  -worker int
    	number of worker (default 4)

Problem solving

error initializing:open apps.json: no such file or directory

At startup, you may receive an error message:

2019/08/11 05:58:02 error initializing:open apps.json: no such file or directory

For its work, the webanalyze program uses the apps.json database file from the Wappalyzer program. The error shown indicates that the file data is missing. To download it, run the command:

webanalyze -update

webanalyze prints less data than wappalyzer

In its work, wappalyzer for determining technologies performs JavaScript code and queries, thanks to which additional information is collected. Since this is not implemented in webanalyze at the moment, it displays less information – only the one that does not require JavaScript code to be executed on the site.