[sunjester tuts] Combolist Management in Bash/Shell Script

[sunjester 1]

Introduction

There seems to be a growing use of combo lists and some people are using horribly compiled lists. This tutorial will help you to create and manage your combo lists better. As with all of my other tutorials, we will be working in Linux, ubuntu xenial. And, yes, backtrack or kali is fine (i belive they are both debian based also). Feel free to use my project, https://github.com/therealsunjester/combos, it hasn't been update in a while but it has over 17 million combo lists, and don't forget to contribute your lists to it.

 

Requirements

All the tools I use are already bundled with a debian distro.

 

Counting the Lists

There is a handy tool called wc, word counter. You can count the lines in files, or even nested folders with files.

 

wc -l 08k.txt

 

As you can see in the image this file has 8,846 lines. We can also check ALL the txt files in the directory.

wc -l *.txt

 

wc prints the line count for each file and the total number of lines counted.

 

Combining Combo Lists

You can use the cat command to merge all the combos into one large combo file. Let's say we have multiple files (I will be using another file from my combo list project on Github).

cat *.txt >combined.txt

And then count the lines in combined.txt and below is both commands, the cat command takes all the txt files and puts them in combined.txt and then we count the lines in combined.txt.

 

Removing Duplicates

It's easy to remove the duplicates using sort.

sort -u combined.txt -o combined.txt

 

As you can see above we removed some duplicates from the list. The original line count in combined.txt was 108,811 and now after the unique sort we have 108,797.

[socalgurl 1]

thanks for this