Programs for generating wordlists

Crunch VS Hashcat

Crunch is a wordlist generator where you can specify a standard character set or a character set you specify. crunch can generate all possible combinations and permutations according to given criteria. The data that crunch outputs can be displayed on the screen, saved to a file, or transferred to another program.

It is a very flexible dictionary generation program.

Hashcat is the world's fastest password cracker, advanced password recovery tool. But in this article we are not interested in this function. Hashcat can generate dictionaries as well. In order not to crack passwords, but only to show candidates in Hashcat, there is an option --stdout.

Hashcat uses various methods for generating passwords (by mask, combinatorial, permutation, Rule-based attack, and others). The most popular way is to generate passwords using a mask.

In order not to get confused in terms, we will call a symbol in a certain place in a mask or pattern (sample) “position”. For example, the first character in password candidates is the first position. The second character is the second position, etc.

By using a mask, we can take control of each position.

For example, we want to make one of the positions a static symbol (always the same in all password candidates) – so can Crunch. But we can also use custom character sets for specific positions. For example, we know that only numbers from 1, 3, 5 are found in the second position. In Hashcat, we can create a custom character set that includes the numbers 1, 3, 5 and put this custom set in the second position in the mask.

In Crunch there is no concept of “mask”, but there is a concept of “pattern”, “sample”. In essence, it is equivalent to a mask. What's more, Crunch can also create up to four custom character sets. Those, in terms of functionality, Crunch is not inferior to the capabilities of Hashcat.

Crunch has a couple of options that have no analogues in Hashcat. For example, -d, it allows you to specify the maximum number of repeated characters. And the -p option creates dictionaries without duplicate characters. Those, this program has its own unique capabilities.

Generating dictionaries with Crunch

Simple usage example:

crunch <minimum-length> <maximum-length> [charset]

For example:

crunch 3 7 abcd

will create all passwords from 3 to 7 characters containing the characters “abcdef” as a character set and print all this to standard output.

The -o option allows you to specify a file to write the output to, for example:

crunch 3 7 abcd -o dict.txt

The -t option allows you to create samples. In Hashcat terms, these are “masks”.

The following abbreviations are used to denote a character set:

  • @ means lowercase letters
  • , means uppercase letters
  • % means numbers
  • ^ means different symbols, 33 in total. You can see them with the command:
crunch 1 1 -t ^

All other characters will output as is.

For example, pattern 8905%%%%%%% means generating password candidates, each starting with 8905 and then followed by seven digits. The complete command will look like this:

crunch 11 11 -t 8905%%%%%%%

Custom Crunch Character Sets

Crunch uses the following character set abbreviations (let's number them):

  • @
  • ,
  • %
  • ^

Any of them can be reassigned. To do this, after specifying the password length range, you need to write down four groups of user characters, if you do not want to reassign a group, put a + (plus) sign instead of your characters. Example:

crunch 2 2 VRb5 + 123 + -t @%

Where:

  • VRb5 is a custom character set, it reassigns @, i.e. instead of abcdefghijklmnopqrstuvwxyz, the abbreviation @ contains the VRb5 character set.
  • The first + sign means that the character set abbreviated as , (comma) and which includes ABCDEFGHIJKLMNOPQRSTUVWXYZ is not reassigned and the default set is used.
  • Next 123 is the second user-defined character set, including the digits 123, respectively, this set reassigns the % abbreviation value
  • The last plus sign indicates that the ^ character set has not been reassigned.

Output:

V1
V2
V3
R1
R2
R3
b1
b2
b3
51
52
53

Let's take another example:

crunch 4 4 + + 123 + -t %%@^

the plus sign (+) is a placeholder, so you can specify a custom character set for any type of set. Crunch will use a specific type of default character set when it encounters a plus sign (+) on the command line. You must either specify values for each type of character set or use the plus sign. That is, if you have two types of character sets, you MUST either provide values for each type or use the plus sign. Those, the previous example will use the following character sets:

  • abcdefghijklmnopqrstuvwxyz
  • ABCDEFGHIJKLMNOPQRSTUVWXYZ
  • 123
  • !@#$%^&*()-_+=~`[]{}|\:;"'<>,.?/ 

there is a space at the end of the above line, the output will start at “11a!” and will end in “33z ”. Quotes indicate a space at the end of a line.

It is recommended that you familiarize yourself with all the options, as well as additional examples of using Crunch: http://en.kali.tools/?p=182

Generating passwords with Hashcat and maskprocessor

If Hashcat does not work for you (it requires the installation of video card drivers), then you can use the maskprocessor program.

Maskprocessor is a high-performance word (dictionary) generator packaged in a separate executable file in which you can customize the characters for each position.

Usage maskprocessor:

maskprocessor [options] ... mask

The mask is composed of a built-in character set and/or a custom character set.

Built-in character sets:

  ?l = abcdefghijklmnopqrstuvwxyz
  ?u = ABCDEFGHIJKLMNOPQRSTUVWXYZ
  ?d = 0123456789
  ?s =  !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
  ?a = ?l?u?d?s
  ?b = 0x00 - 0xff

All characters, except for those that stand for the set (?l, ?u, ?d, etc.), are included in the password unchanged.

If you want to compose a dictionary that contains six-digit passwords:

maskprocessor ?d?d?d?d?d?d

If you want to alternate uppercase and lowercase letters in four-character passwords:

maskprocessor ?u?l?u?l

The password starts with voro, followed by four numbers, and then two capital letters:

maskprocessor voro?d?d?d?d?u?u

Custom character sets:

  -1,  --custom-charset1=CS  User-defineable charsets
  -2,  --custom-charset2=CS  Example:
  -3,  --custom-charset3=CS  --custom-charset1=?dabcdef
  -4,  --custom-charset4=CS  sets charset ?1 to 0123456789abcdef

For example, in the first position there should be numbers from 1 to 3, in the second position of the letter AbCd, in the third position any number or lowercase letter, and the last two characters should be uppercase letters:

maskprocessor -1 123 -2 AbCd -3 ?l?d ?1?2?3?u?u

Here:

  • -1 123 is the first custom character set including characters 123
  • -2 AbCd is the second custom character set including AbCd characters
  • -3 ?l?d is the third user-defined character set that includes the characters? L? D, i.e. all small letters and numbers
  • ?1?2?3?u?u is a mask composed of custom and standard character sets.

Additional useful options:

* Increment:

  -i,  --increment=NUM:NUM   Enable increment mode. 1st NUM=start, 2nd NUM=stop
                             Example: -i 4:8 searches lengths 4-8 (inclusive)

* Misc:

       --combinations        Calculate number of combinations
       --hex-charset         Assume charset is given in hex
  -q,  --seq-max=NUM         Maximum number of multiple sequential characters
  -r,  --occurrence-max=NUM  Maximum number of occurrence of a character

* Resources:

  -s,  --start-at=WORD       Start at specific position
  -l,  --stop-at=WORD        Stop at specific position

* Files:

  -o,  --output-file=FILE    Output-file

Hashcat VS maskprocessor

Although, in general, maskprocessor and Hashcat are interchangeable to generate passwords, remember that the -a 3 option must be specified to select the brute-force/mask attack mode (since Hashcat supports various attack modes, not just mask). You also need to use the --stdout option, which means to show the password candidates (without cracking the hash).

Hashcat (Mask attack) does not allow you to set the maximum number of identical repeated characters, the maximum number of occurrences of one character, start or end at a specific position. But such a result can be obtained using a Rule-based attack.

Running Hashcat on Linux systems can be problematic due to the need to have proprietary drivers.

The increment of the password length is done by other options:

 Options Short / Long           | Type | Description                                          | Example
================================+======+======================================================+=======================
 -i, --increment                |      | Enable mask increment mode                           |
     --increment-min            | Num  | Start mask incrementing at X                         | --increment-min=4
     --increment-max            | Num  | Stop mask incrementing at X                          | --increment-max=8

How to generate wordlists with John the Ripper

John the Ripper also supports the generation of wordlists by mask. Moreover, the syntax is very similar to the syntax of masks in Hashcat. But there is also a difference: John the Ripper is more flexible, it allows using ranges of charsets when writing masks, any characters in hexadecimal notation, it has more standard character sets.

A mask may consist of:

  • Static letters.
  • Ranges in [aouei] or [a-z] syntax. Or both, [0-9abcdef] is the same as [0-9a-f].
  • Placeholders that are just a short form for ranges, like ?l which is 100% equivalent to [a-z].
  • ?l lower-case ASCII letters
  • ?u upper-case ASCII letters
  • ?d digits
  • ?s specials (all printable ASCII characters not in ?l, ?u or ?d)
  • ?a full 'printable' ASCII. Note that for formats that don't recognize case (eg. LM), this only includes lower-case characters which is a tremendous reduction of keyspace for the win.
  • ?B all 8-bit (0x80-0xff)
  • ?b all (0x01-0xff) (the NULL character is currently not supported by core).
  • ?h lower-case HEX digits (0-9, a-f)
  • ?H upper-case HEX digits (0-9, A-F)
  • ?L lower-case non-ASCII letters
  • ?U upper-case non-ASCII letters
  • ?D non-ASCII "digits"
  • ?S non-ASCII "specials"
  • ?A all valid characters in the current code page (including ASCII). Note that for formats that don't recognize case (eg. LM), this only includes lower-case characters which is a tremendous reduction of keyspace.
  • Placeholders that are custom defined, so we can e.g. define ?1 to mean [?u?l]
  • ?1 .. ?9 user-defined place-holder 1 .. 9
  • Placeholders for Hybrid Mask mode:
  • ?w is a placeholder for the original word produced by the parent mode in Hybrid Mask mode.
  • ?W is just like ?w except the original word is case toggled (so PassWord becomes pASSwORD).

Mask Mode alone produces words from the mask, for example ?u?l?l will generate all possible three-letter words, with first character uppercased and the remaining in lowercase.

Variable length masks

There is a default mask in john.conf too (defaulting to same as hashcat). This should be used with -max-len (and possibly -min-len) to do any good.

The -max-len=N option will truncate the mask so no words longer than N are produced.

The -min-len=N option will skip generation of words shorter than N.

If not in “hybrid mask” mode, and either -min-len or -max-len option was used, we will iterate lengths (as in “incremental mask”) from -min-len to -max-len (or format's min or max length, if one was not given). So to produce all possible words from 3 to 5 letters, use -mask=?l -min-len=3 -max-len=5. In case the specified mask is shorter, the last part of it will be expanded, for example “-mask=?u?l -max-len=5” will use an effective mask of ?u?l?l?l?l. Whenever using incremental mask, the ETA at any given time shows estimated time to complete the current length, as opposed to the whole run.

Special characters in masks

You can escape special characters with \. So to produce a literal “?l” you could say \?l or ?\l and it will not be parsed as a placeholder. Similarly, you can escape dashes or brackets to prevent them from being parsed as specials. To produce a literal backslash, use \\.

There is also a special hex notation, \xHH for specifying any character code. For example, \x41 is “A” and \x09 is the code for TAB.

Examples:

Mask Custom mask / hybrid input Output Num candidates
pass   pass 1
pw?d   pw3 10
?w?d?d?d password password123 1000x
?w?s?w Bozo Bozo#Bozo 33x
?w?s?W Bozo Bozo#bOZO 33x
0x?1?1:?1?1:?1?1 -1=[0-9a-f] 0xde:ad:ca 16777216
?3?l?l?l -3=?l?u Bozo, hobo 913952
[Pp][Aa@][Ss5][Ss5][Ww][Oo0][Rr][Dd]   P@55w0rD 1296

We have on-device mask mode support for most fast hash types for which we have OpenCL support at all. Further, such on-device mask support can be used along with a host-provided stream of partial candidate passwords to form a variety of hybrid modes. For example, these are all valid:

  • Test any 7-character printable ASCII strings, with a reasonable number of the mask positions being processed on device (JtR decides to optimally split the mask between host and device):
--mask='?a?a?a?a?a?a?a'

Ditto:

--mask='?a' --min-length=7 --max-length=7

Ditto, but for range of lengths 1 to 8:

--mask='?a' --min-length=1 --max-length=8

Can also use length ranges with more complex masks, where the last mask component would be the one extended to higher lengths:

--mask='start?l?d' --min-length=7 --max-length=14

Other, more complex examples of using the mask will be considered in a separate part, which is completely devoted to the modes of generating, changing and processing passwords.

How to view generated passwords. How to create a dictionary

With the --stdout option, you can show the generated password candidates instead of starting the cracking. This can be useful when checking composed masks and when generating dictionaries.

For example, to show all the password candidates for the '?d?d?d?d' mask:

./john --mask='?d?d?d?d' --stdout

To save all passwords for mask '?d?d?d?d' (all four digits words) to a file:

./john --mask='?d?d?d?d' --stdout > 4d.txt

To create a dictionary containing all numbers from 1 to 9999 and save it to 1-4d.txt file:

./john --mask='?d?d?d?d' --min-length=1 --max-length=4 --stdout > 1-4d.txt

To create a file with passwords (phone numbers) starting with “8905143”:

./john --mask='8905143?d?d?d?d' --stdout > beeline.txt

You can print out only password candidates limited by LENGTH, use an option like--stdout=LENGTH.

Conclusion

Crunch is a very flexible dictionary generating program. On the downside, we can note that the assignment of a custom character set is somewhat confusing, and that custom character sets overwrite the standard ones.

Hashcat does not have some options related to regulating the number of repeated characters in a password.

maskprocessor is a good alternative to Crunch. The program completely repeats the functionality of Crunch, while allowing you to assign custom character sets without overwriting the standard sets. The assignment syntax itself seems to be more descriptive. Hashcat and maskprocessor run on both Linux and Windows.

John the Ripper includes all of Hashcat's wordlist generation capabilities, but also has additional features, some of which may be of interest to professionals.

Recommended for you:

Leave a Reply

Your email address will not be published. Required fields are marked *