Monday 27 September 2010

Wordlist Sizes

The post on creating wordlists with crunch v2.4 receives the most hits by far on my blog and from the
queries in the comments section, it would seem that not everyone realises what the potential size can be
when creating wordlists.

EDIT
====
Check out the latest revision of crunch, bofh28 just released v2.6 03-10-2010.
Crunch is now including a size estimate when starting up the wordlist generation, so you can see what size the wordlist you are planning will be.
That along with a few more new nice additions.
Download the latest crunch here;
http://sourceforge.net/projects/crunch-wordlist/
Edit
latest revision of crunch now also included in the backtrack 4 repository.


Lets say you are working on a wordlist for a WPA key (which always have a minimum of 8 characters)
and lets say that you know for a fact that the passkey in question is an eight character combination of the following digits and letters;
0123456789ABCDEF
(like some internet companies have on their broadband modem/routers where I am from).

To create a wordlist with all possible combinations based on the passphrase having 8 characters only,
you could use the following syntax in crunch;

./crunch 8 8 0123456789ABCDEF -o wpa-list.txt

That one line of code seems so simple, yet when you check the estimated size of the wordlist to be created
you would definately think twice about trying to create, save and use it...


The size of the wordlist can be calculated as follows ;

(x^y) * (y+1) = size in bytes
x = The number of characters being used to create the wordlist
y = The number of characters the words/passphrases in the wordlist have.

Based on the above example, we have 10 possible numeric values and 6 possible alpha values,
so 16 characters in total, and we want to calculate based on a wordlist wherein the passphrases have 8 characters.
To calculate what the size would be in konsole we can use "bc" ;

echo "(16^8)*(8+1)" | bc




Or we can even just type it in google; (16^8)*(8+1)
and it will return the same result ;
















Next we can check the conversions of the resulting size in KB / MB / GB etc. ;










thats quite a lot...


I put together a (very!) simple script in order to be able to quickly check what kind of size one
is looking at when thinking of creating a wordlist with the same min/max length in crunch;
crunch_size

DOWNLOAD
http://www.mediafire.com/file/dmh989dhmebch43/crunch_size-v0.2

After saving to your /root/ directory for instance, just run by entering ;

./crunch_size-v0.2

You need to enter ;
> the number of characters to be used when creating the wordlist. (using the above example; 16)
> the length of the words/passphrases in the wordlist. (using the above example; 8)



















You cant choose to check what the results would be with any fixed patterns, or variables, (have to leave the hard stuff like that to the pro's !) but it is still an eye-opener to see the sizes involved with a 'simple' wordlist.


The result will show you the expected number of words/passphrases in the wordlist along with the estimated
file size in bytes / Kilobytes / Megabytes / Gigabytes / Terabytes / Petabytes






















Just a bit of fun and possibly handy to have in your crunch directory for reference ;)

Please comment if I messed up on the calculations anywhere..

17 comments:

  1. You can do your Bytes to GigaBytes (or whatever) conversion right along with your google calculation, simply use "(16^8)*(8+1) bytes to gigabytes" as your search term.

    ReplyDelete
  2. Ha !

    Didnt know that .. works a treat,
    thanks for the tip !

    ReplyDelete
  3. Hi!

    It is possible to create a wordlist with crunch v2.4 with just only birthdays in the range and in format 01011900-01012100 . What syntax should i enter in crunch?

    ReplyDelete
  4. Hey maersk,

    No, crunch is not able to do what you want.

    To do that you would need to write a bit of code.
    If I have a moment later I will have a look.

    ReplyDelete
  5. Had a look and my little grey cells could not figure out a nice one-liner.
    But got somewhere with the below code.

    Copy and paste below code and save as datelist.
    Make executable by doing;
    chmod 755 datelist
    Then execute the coe by doing;
    ./datelist

    Let me know if the below code semi-works for you !



    #!/bin/bash
    #
    #
    # Create a wordlist based on dates in the format
    # ddmmyyyy
    # date doesnt seem to like certain dates before 1940..
    # so test as from 1941 ;)
    #
    # Although the input needs to be yyyy-mm-dd
    # the output with the below script will be ddmmyyyy
    # Its slow and suppose it would be quicker if it wasnt
    # 'teed' to screen, but what else are you gonna stare at..
    #
    echo "Enter the starting date"
    echo "must be in the format yyyy-mm-dd"
    (tput bold && tput setaf 1)
    read START_DATE
    (tput sgr 0)
    echo "Enter the ending date"
    echo "must be in the format yyyy-mm-dd"
    (tput bold && tput setaf 1)
    read END_DATE
    (tput sgr 0)
    # List all dates in between the above dates

    echo $START_DATE | tee r_dates.txt
    while true
    do
    START_DATE=$( date +%Y-%m-%d -d "$START_DATE -d 1day" )
    echo $START_DATE | tee -a r_dates.txt
    if [ "$START_DATE" == "$END_DATE" ]
    then
    awk -F- '{print $3 $2 $1}' r_dates.txt > datelist.txt
    rm r_dates.txt
    (tput bold && tput setaf 1)
    echo "wordlist 'datelist.txt' created"
    (tput sgr 0)
    exit
    fi
    done

    ReplyDelete
  6. Further to the above, there are some issues with certain dates, for instance in 2038.
    http://2038bug.com/

    Otherwise the above code 'seems' to work OK with probably a couple of errors..

    ReplyDelete
  7. Thank's a lot!!! Tested in BT4. It works!!!
    Tnan i can play with syntax "+%Y-%m-%d" getting different format of date.
    Thank's again. :)

    ReplyDelete
  8. Yeah it took me a while to figure something out, I had sorta hoped it would be easier ..
    But the little grey cells were failing me ;)

    Play with the +%Y-%m-%d & the awk setting to change the date formatting around.

    Glad it was of help :D

    ReplyDelete
  9. Can a word list be made with the prefix "elite" merged with every word in a dictionary list?
    ex: elitea eliteaback eliteabacus...

    ReplyDelete
  10. Sure, cat out the wordlist and then use sed to append / prepend a word ;

    To prepend;
    cat wordlist.txt | sed 's/^/elite/' > newlist.txt

    To append;
    cat wordlist.txt | sed 's/$/elite/' > newlist.txt

    ReplyDelete
  11. Thanks,your commands worked.
    Is there a way to capitalize the first letter of the appended word?
    ex: eliteA...eliteAback...eliteAbacus...
    Also: How would a space be added?
    ex: elite A...elite Aback...elite Abacus...

    ReplyDelete
  12. Glad it helped, you will have to look at how to use sed commands to alter a wordlist to your liking when considering capitalizing the first letter.

    Have a read through the comments posted in the blogpost on using crunch v2.4 (April 2010), there is some useful information in that on using sed to capitalize the first and last letter.

    ReplyDelete
  13. I love the script keep it up. Nice tutorial.

    ReplyDelete
  14. Thanks :)

    To be honest since crunch v2.6 came, no real use for this script as crunch will show you the size
    of the wordlist to be created.

    But it was all a bit of fun :)

    ReplyDelete
  15. Hello TAPE... excelent post and ur script é very simple and good!
    I also have a blog that talk about security, R&S and Wireless and I would like to know if you allow me to translate ur script to Brazilian Portuguese.

    Tks
    Marco Bartulihe

    ReplyDelete
    Replies
    1. Sure thing bartulihe, go for it ;)

      Would appreciate you link to this blog should you do post about it ;)

      Delete
  16. Hi tape,

    How I can make the word list with abcdefghijklmnopqrstuvxzwy ABCDEFGHIJKLMNOPQRSTUVXZWY 0123456789 @#$% ?

    With 8 8 and split in 4 files or more?

    Thanks

    ReplyDelete

 
Google Analytics Alternative