Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sensepost/common-substr
Simple tool to extract the most common substrings from an input text. Built for password cracking.
https://github.com/sensepost/common-substr
cracking passwords string-manipulation wordlist-generator
Last synced: about 1 month ago
JSON representation
Simple tool to extract the most common substrings from an input text. Built for password cracking.
- Host: GitHub
- URL: https://github.com/sensepost/common-substr
- Owner: sensepost
- License: gpl-3.0
- Created: 2018-05-13T20:07:20.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-06-28T20:18:21.000Z (over 4 years ago)
- Last Synced: 2024-11-02T18:41:34.807Z (about 1 month ago)
- Topics: cracking, passwords, string-manipulation, wordlist-generator
- Language: Go
- Homepage:
- Size: 42 KB
- Stars: 61
- Watchers: 7
- Forks: 18
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-password-cracking - common-substr - Simple tool to extract the most common substrings from an input text. Built for password cracking. (Wordlist tools / Generation/Manipulation)
README
# common-substr
Simple tool to extract the most common substrings from an input text. Built for password cracking. A write-up on the theory can be found [at the sensepost.com blog](https://sensepost.com/blog/2018/cracking-efficiency-measurements-common-substring-attack/)
There are two versions, the older awk script and the newer & faster golang version. They use the same commandline switches.
# Usage
```
Common Substring Generator by @singe
Usage: ./common-substr [-hinsp] [-t ] [-l ] [-L ] -f
-h|--help This help
-i|--insensitive Ignore case of substrings
-L|--maxlength Maximum length substring to look for. Default is 32.
-l|--minlength Minimum length substring to look for. Default is 2.
-n|--nostats Just print the substrings, no stats. Default is to include them.
-t|--threshold Only print substrings more prevalent than percent.
-f|--file The file to extract substrings from
-s|--suffix Only look at suffix substrings at the end of a string
-p|--prefix Only look at prefix substrings at the beginning of a string
Default output (with stats) is tab separated:
Sorted from most to least common
```
## Simple Usage Examples
Given the test file:
```
123
123
234
```
We can find the most common substrings:
```
./common-substr -f test
100 3 23
66.6667 2 12
66.6667 2 123
```
Read this output as "100% of the input file had the substring "23" which consisted of 3 instances".
Do the same, but suppress printing of the stats:
```
./common-substr -f test -n
23
12
123
```
Only include substrings that occur at least 70% of the time:
```
./common-substr -f test -t 70
100 3 23
```
The stats are tab-separated, to make cut'ing easy:
```
./common-substr -f test > output
cut -f 3 output
23
12
123
```
Only include substrings 3 characters or longer:
```
./common-substr -f test -l 3
66.6667 2 123
```
Only include substrings 2 characters or shorter:
```
./common-substr -f test -L 2
100 3 23
66.6667 2 12
```
Only include the start of the strings (prefix):
```
./common-substr -f test -p
66.6667 2 12
66.6667 2 123
```
Only include the end of the strings (suffix):
```
./common-substr -f test -s
66.6667 2 23
66.6667 2 123
```
# Password Cracking Examples
## Vanilla wordlist + substrings
An example use for password cracking. Assuming you've put already cracked clear-text passwords in a file called 'passwords':
```
# Limit substrings to a max length of 27 and only include those which occur
# at least 1% or more of the time
./common-substr -t 1 -l 27 -n -f passwords > substrs
sort -u passwords > uniques
hashcat -a1 hashes uniques substrs
```
## Basewords + substrings
It also helps to create "base words" and combine those with the substrings:
```
grep -oi "[a-z]*[a-z]" uniques > basewords
hashcat -a1 hashes basewords substrs
```
Remember to try it the other way around too:
```
hashcat -a1 hashes substrs basewords
```
## All Substrings
Drop the threshold and throw the full list of substrings into combinator:
```
./common-substr -n -f passwords > all-substrs
hashcat -a1 hashes all-substrs all-substrs
```
## Prefix & Suffix Substrings
Take the commons starts and ends of passwords and combine them:
```
./common-substr -n -p -f passwords > prefix
./common-substr -n -s -f passwords > suffix
hashcat -a1 hashes prefix suffix
```
# Building
The golang version can be built using `go build ./common-substr.go`.
The awk version can be run using the `common-substr.sh` wrapper script. It requires awk.
I recommend the golang version.