Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/sensepost/common-substr

Simple tool to extract the most common substrings from an input text. Built for password cracking.
https://github.com/sensepost/common-substr

cracking passwords string-manipulation wordlist-generator

Last synced: 3 months ago
JSON representation

Simple tool to extract the most common substrings from an input text. Built for password cracking.

Host: GitHub
URL: https://github.com/sensepost/common-substr
Owner: sensepost
License: gpl-3.0
Created: 2018-05-13T20:07:20.000Z (about 6 years ago)
Default Branch: master
Last Pushed: 2020-06-28T20:18:21.000Z (about 4 years ago)
Last Synced: 2024-01-24T23:45:54.487Z (6 months ago)
Topics: cracking, passwords, string-manipulation, wordlist-generator
Language: Go
Homepage:
Size: 42 KB
Stars: 62
Watchers: 7
Forks: 18
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Lists

awesome-password-cracking - common-substr - Simple tool to extract the most common substrings from an input text. Built for password cracking. (Wordlist tools / Generation/Manipulation)

README

# common-substr
Simple tool to extract the most common substrings from an input text. Built for password cracking. A write-up on the theory can be found [at the sensepost.com blog](https://sensepost.com/blog/2018/cracking-efficiency-measurements-common-substring-attack/)

There are two versions, the older awk script and the newer & faster golang version. They use the same commandline switches.

# Usage
```
Common Substring Generator by @singe
Usage: ./common-substr [-hinsp] [-t ] [-l ] [-L ] -f
-h|--help This help
-i|--insensitive Ignore case of substrings
-L|--maxlength Maximum length substring to look for. Default is 32.
-l|--minlength Minimum length substring to look for. Default is 2.
-n|--nostats Just print the substrings, no stats. Default is to include them.
-t|--threshold Only print substrings more prevalent than percent.
-f|--file The file to extract substrings from
-s|--suffix Only look at suffix substrings at the end of a string
-p|--prefix Only look at prefix substrings at the beginning of a string
Default output (with stats) is tab separated:
Sorted from most to least common
```

## Simple Usage Examples

Given the test file:
```
123
123
234
```

We can find the most common substrings:
```
./common-substr -f test
100 3 23
66.6667 2 12
66.6667 2 123
```
Read this output as "100% of the input file had the substring "23" which consisted of 3 instances".

Do the same, but suppress printing of the stats:
```
./common-substr -f test -n
23
12
123
```

Only include substrings that occur at least 70% of the time:
```
./common-substr -f test -t 70
100 3 23
```

The stats are tab-separated, to make cut'ing easy:
```
./common-substr -f test > output
cut -f 3 output
23
12
123
```

Only include substrings 3 characters or longer:
```
./common-substr -f test -l 3
66.6667 2 123
```

Only include substrings 2 characters or shorter:
```
./common-substr -f test -L 2
100 3 23
66.6667 2 12
```

Only include the start of the strings (prefix):
```
./common-substr -f test -p
66.6667 2 12
66.6667 2 123
```

Only include the end of the strings (suffix):
```
./common-substr -f test -s
66.6667 2 23
66.6667 2 123
```

# Password Cracking Examples

## Vanilla wordlist + substrings
An example use for password cracking. Assuming you've put already cracked clear-text passwords in a file called 'passwords':
```
# Limit substrings to a max length of 27 and only include those which occur
# at least 1% or more of the time
./common-substr -t 1 -l 27 -n -f passwords > substrs
sort -u passwords > uniques
hashcat -a1 hashes uniques substrs
```

## Basewords + substrings

It also helps to create "base words" and combine those with the substrings:
```
grep -oi "[a-z]*[a-z]" uniques > basewords
hashcat -a1 hashes basewords substrs
```
Remember to try it the other way around too:
```
hashcat -a1 hashes substrs basewords
```

## All Substrings

Drop the threshold and throw the full list of substrings into combinator:
```
./common-substr -n -f passwords > all-substrs
hashcat -a1 hashes all-substrs all-substrs
```

## Prefix & Suffix Substrings

Take the commons starts and ends of passwords and combine them:
```
./common-substr -n -p -f passwords > prefix
./common-substr -n -s -f passwords > suffix
hashcat -a1 hashes prefix suffix
```

# Building

The golang version can be built using `go build ./common-substr.go`.

The awk version can be run using the `common-substr.sh` wrapper script. It requires awk.

I recommend the golang version.