{"id":13531738,"url":"https://github.com/0xVavaldi/gramify","last_synced_at":"2025-04-01T20:30:29.832Z","repository":{"id":54350648,"uuid":"485409674","full_name":"0xVavaldi/gramify","owner":"0xVavaldi","description":"Create n-grams of wordlists based on words, characters, or charsets to use in offline password attacks and data analysis","archived":false,"fork":false,"pushed_at":"2023-10-06T21:39:35.000Z","size":87,"stargazers_count":26,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-05-21T13:49:45.132Z","etag":null,"topics":["hashcat","jtr","mdxfind","ngram","password","password-analysis","password-cracking"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0xVavaldi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-25T14:39:18.000Z","updated_at":"2024-06-27T11:07:23.598Z","dependencies_parsed_at":"2022-08-13T13:00:38.531Z","dependency_job_id":"cac85d59-5045-498f-9e64-0ad519e653e8","html_url":"https://github.com/0xVavaldi/gramify","commit_stats":null,"previous_names":["0xvavaldi/gramify"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xVavaldi%2Fgramify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xVavaldi%2Fgramify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xVavaldi%2Fgramify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0xVavaldi%2Fgramify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0xVavaldi","download_url":"https://codeload.github.com/0xVavaldi/gramify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246709921,"owners_count":20821296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hashcat","jtr","mdxfind","ngram","password","password-analysis","password-cracking"],"created_at":"2024-08-01T07:01:05.265Z","updated_at":"2025-04-01T20:30:29.468Z","avatar_url":"https://github.com/0xVavaldi.png","language":"Python","funding_links":[],"categories":["Wordlist tools"],"sub_categories":["Generation/Manipulation"],"readme":"# Gramify\n\nGramify is an analysis tool built with the purpose of enhancing attacks on complex passwords through the use of n-grams.\nGramify offers three types of n-gram analysis\n- Word\n- Character\n- Charset\n\nThese each perform n-grams at their respective levels.\n\n## What is an n-gram?\nThose unfamiliar with the term will most easily understand it as the n words that follow each other naturally. At a word level the sentence: \"I am writing a program\" can be split at 2-gram level into: `[\"I am\", \"am writing\", \"writing a\", \"a program\"]`. at 3-gram level into: `[\"I am writing\", \"am writing a\", \"writing a program\"]`. This can also be done at a character level for example with \"abc defg\" into the 3-gram `[\"abc\", \"bc \", \"c d\", \" de\", \"def\", \"efg\"]`. Logically you can imagine that using this on books, or song lyrics can turn into a powerful analytical form where you can extract quotes or find words commonly used together such as the words: \"I am\", \"He is\" instead of: \"capricorn icecream\".\n\n## Word n-grams\nThe options offered by the gramify as of version 0.8 are as follows:\n\n```gramify.py word \u003cinput_file\u003e \u003coutput_file\u003e [--min-length=\u003cint\u003e] [--max-length=\u003cint\u003e]```\n--min-length refers to the minimum amount of words that ngrams should be. Ergo. at least `\u003cint\u003e` words (Default: 1)\n--max-length refers to the maximum amount of words that ngrams should be. Ergo. at least `\u003cint\u003e` words (Default: 10)\n\nExpecting long quotes or lyrics? Increase the --max-length, the penalty is often minor.\n\nExample of input file format:\n```\nBut now that I'm home feels like I'm in heaven\nSee I been travelin'\nOoh, whoah ooh whoah\nI'm in heaven\nOh and I'm, feeling right at home\nFeeling right at home\nFeeling like I'm in heaven\nIt's like I'm in heaven\nIt's like I'm in heaven, ooh we oh\n```\n\nOutput is unsorted data containing duplicates like the word \"It's\" and \"I'm\" as 1-gram or \"I'm in heaven\" as 3-gram.\nSorting them as recommended (sort by occurrence) will put the best at the top, so you can HEAD the output file if the data is too much.\nRead more: https://en.wikipedia.org/wiki/N-gram\n\nSome recommended commands would be:\n```\ngramify.py word \u003cinput_file\u003e \u003coutput_file\u003e \ngramify.py word \u003cinput_file\u003e \u003coutput_file\u003e --max-length=32\n\nhashcat -a0 -m0 \u003chashlist\u003e \u003coutput_file\u003e -r \u003csentence_mangling_rules\u003e\nhashcat -a1 -m0 \u003chashlist\u003e \u003coutput_file\u003e \u003coutput_file\u003e -j\"$ \"\n```\n\n## Character n-grams (k-grams)\nThe options offered by the gramify as of version 0.8 are as follows:\n\n```gramify.py character \u003cinput_file\u003e \u003coutput_file\u003e [--min-length=\u003cint\u003e] [--max-length=\u003cint\u003e] [--rolling]```\n--min-length refers to the minimum amount of characters that ngrams should be. Ergo. at least `\u003cint\u003e` characters (Default: 4)\n--max-length refers to the maximum amount of characters that ngrams should be. Ergo. at least `\u003cint\u003e` characters (Default: 8, or 32 if rolling)\n--rolling explained later\n\nk-grams or character-based n-grams are great for analyzing what passwords start or end with. Therefore it is split up into start_ mid_ and end_.\nThe start_ is ideal with hashcat -a6 using it as the input dictionary with mask appended to it\nOn the opposite end is the end_ which is great with hashcat -a7 using it as the end of the word with a mask prepended to it.\nAdditionally you can use start_, mid_ and end_ in any combination using -a1 or combinatorX.exe (from hashcat-utils) to combine them back together in any combination available, resulting in probable passwords.\n\nThe start_ will contain `^.{--min-length, --max-length}` or everything from the start of the line until everything between --min-length and --max-length.\nBy default this would be the regex: `^.{4,8}`\n\nThe end_ will contain `.{--min-length, --max-length}$` or everything from the start of the line until everything between --min-length and --max-length.\nBy default this would be the regex: `.{4,8}$`\n\nThe mid_ section will contain the remainder of whatever is availabe: Seeing start_, mid_ and end_ as separate regex groups you could represent it as this: `^(.{--min-length, --max-length})(.*?)(.{--min-length, --max-length})$`\n\n--rolling addresses some of the limitations this has. The benefit of having them split is that you have 3 different parts that each have a specific function. But sometimes you're not looking for the specific start, mid, end but more the classic k-gram as specified before. This would be it. It produces one file that has character-based ngram for all lengths.\n\n\nSome recommended commands would be:\n```\ngramify.py character \u003cinput_file\u003e \u003coutput_file\u003e \ngramify.py character \u003cinput_file\u003e \u003coutput_file\u003e --max-length=128           (this would essentially empty out the mid_ file.\ngramify.py character \u003cinput_file\u003e \u003coutput_file\u003e --rolling\n\nhashcat -a0 -m0 \u003chashlist\u003e start_\u003coutput_file\u003e mid_\u003coutput_file\u003e end_\u003coutput_file\u003e -r \u003cpopular rules\u003e\nhashcat -a0 -m0 \u003chashlist\u003e start_\u003coutput_file\u003e mid_\u003coutput_file\u003e end_\u003coutput_file\u003e -r \u003ctop_1500.rule\u003e -r \u003ctop_1500.rule\u003e\nhashcat -a6 -m0 \u003chashlist\u003e start_\u003coutput_file\u003e ?a?a?a?a -i\nhashcat -a7 -m0 \u003chashlist\u003e ?a?a?a?a end_\u003coutput_file\u003e -i\nhashcat -a1 -m0 \u003chashlist\u003e start_\u003coutput_file\u003e mid_\u003coutput_file\u003e\nhashcat -a1 -m0 \u003chashlist\u003e start_\u003coutput_file\u003e end_\u003coutput_file\u003e\nhashcat -a1 -m0 \u003chashlist\u003e mid_\u003coutput_file\u003e start_\u003coutput_file\u003e\nhashcat -a1 -m0 \u003chashlist\u003e mid_\u003coutput_file\u003e end_\u003coutput_file\u003e\nhashcat -a1 -m0 \u003chashlist\u003e end_\u003coutput_file\u003e start_\u003coutput_file\u003e\nhashcat -a1 -m0 \u003chashlist\u003e end_\u003coutput_file\u003e mid_\u003coutput_file\u003e\n```\nRead more: https://nlp.stanford.edu/IR-book/html/htmledition/k-gram-indexes-for-wildcard-queries-1.html\n\n## Charset n-grams\n`gramify.py charset \u003cinput_file\u003e \u003coutput_file\u003e [--min-length=\u003cint\u003e] [--max-length=\u003cint\u003e] [--mixed]`\n\n--min-length refers to the minimum amount of characters that each word should have. Ergo. at least `\u003cint\u003e` characters (Default: 4)\n--max-length refers to the maximum amount of characters that each word should have. Ergo. at least `\u003cint\u003e` characters (Default: 32)\n--mixed do not make a distinction between upper and lowercase or upper, lowercase and numeric in two different passes\n--filter make use of the start, mid and end filters to combine and only grab the first element, mid elements, or last element.\n\n\nThis type of n-gram is more focused on character set boundries. Moving from UPPERCASE to lowercase. From Digits to lowercase or vice versa. This way you're able to take the passwords (assuming a default --min-length of 4):\n```\npassword123456 -\u003e [password, 123456]\nPASSword123123magicman -\u003e [PASS, word, 123123, magicman]\nTHEBESTINTHEWORLD54321 -\u003e [THEBESTINTHEWORLD, 54321]\nthere are a lot of things to say -\u003e [there, things]\n```\nThis can be great for extracting words or common patterns out of passwords, removing punctuation or discovering common themes. From here on we can use rules on our newly discovered words to find new passwords. Gramify builds on this concept by allowing an Uppercase character to go to a lowercase character, but only if it's the first in the word allowing the capture of items like `PassWord123456 -\u003e [Pass, Word, 123456]`.\n\nAn example of --filter using the above examples and the following available filters: solo, start, mid, and end, startmid, midend, startend:\n```\npython3 gramify.py charset hashmob.net_2022-07-03.found test.txt --filter 'start,mid,end,startmid,midend'\n\npassword123456 will be converted into [password, 123456] which will have:\npassword =\u003e start\npassword123456 =\u003e startend\n123456 =\u003e end\n\nPASSword123123magicman will be converted into [PASS, word, 123123, magicman] which will have:\nPASS =\u003e start\nword123123 =\u003e mid\nmagicman =\u003e end\nPASSword123123 =\u003e startmid\nword123123magicman =\u003e midend\n\npassword will be converted into [password]: which falls under 'solo' and therefore isn't written to a file as it's not part of our filters.\n```\n\nIf you want even more options: using the --mixed will help with short words with many upper and lowercase like:\n```\nPaSSwOrd123123 -\u003e [PaSSwOrd, 123123]\n```\nThese will come in addition to the other passwords. Current settings do not allow for exclusive mixed generation.\n\nInspired by: https://github.com/hops/pack2 (https://github.com/hops/pack2/blob/master/src/cgrams.rs)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xVavaldi%2Fgramify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0xVavaldi%2Fgramify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0xVavaldi%2Fgramify/lists"}