{"id":26541585,"url":"https://github.com/peter-juhasz/wordlist-tool","last_synced_at":"2025-03-22T01:39:15.178Z","repository":{"id":89618085,"uuid":"597669614","full_name":"Peter-Juhasz/wordlist-tool","owner":"Peter-Juhasz","description":"Chainable, high-performance tool for manipulating wordlists.","archived":false,"fork":false,"pushed_at":"2024-06-18T07:06:17.000Z","size":78,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-06-18T08:29:20.223Z","etag":null,"topics":["high-performance","password-cracking","wordlist"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Peter-Juhasz.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-05T09:11:27.000Z","updated_at":"2024-06-18T08:29:22.251Z","dependencies_parsed_at":null,"dependency_job_id":"e76baa47-578c-445b-b37c-4ba0069c3802","html_url":"https://github.com/Peter-Juhasz/wordlist-tool","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter-Juhasz%2Fwordlist-tool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter-Juhasz%2Fwordlist-tool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter-Juhasz%2Fwordlist-tool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Peter-Juhasz%2Fwordlist-tool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Peter-Juhasz","download_url":"https://codeload.github.com/Peter-Juhasz/wordlist-tool/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244894308,"owners_count":20527669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["high-performance","password-cracking","wordlist"],"created_at":"2025-03-22T01:39:14.598Z","updated_at":"2025-03-22T01:39:15.162Z","avatar_url":"https://github.com/Peter-Juhasz.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Wordlist Tool\nA chainable, high-performance tool for manipulating wordlists. Wordlists can be used for many purposes: word games, allow or deny lists, even for password cracking.\n\n\n## Usage\n```\nUsage:\n  wl [command] [options]\n\nOptions:\n  --encoding \u003cencoding\u003e        Default encoding of the wordlist file. [default: ASCII]\n  --line-ending \u003cline-ending\u003e  Default line ending sequence. [default: 0A]\n  --buffer-size \u003cbuffer-size\u003e  Default buffer size for reading and writing. [default: 16384]\n  --version                    Show version information\n  -?, -h, --help               Show help and usage information\n\nCommands:\n  transform       Transform entries.\n  filter          Filter entries.\n  sort            Sort entries.\n  list            Transform list.\n  merge           Merge entries from multiple lists.\n  split           Split a list into multiple lists.\n  extract         Extract entries from other formats.\n  generate        Generate entries.\n```\n\nFor example:\n```ps\nwl extract --inputs books/*.txt --output OUT --regex \\w+ |\nwl filter distinct IN OUT |\nwl sort asc IN OUT |\nwl transform-list take IN output.txt 1000\n```\n\n\n### Sort\nYou can sort entries in ascending order the following way:\n```ps\nwl sort entries list.txt\n```\n\nOutput can be saved to another file:\n```ps\nwl sort entries list.txt output.txt\n```\n\nSort entries in a descending way:\n```ps\nwl sort entries list.txt --descending\n```\n\nSort entries by length:\n```ps\nwl sort length list.txt\n```\n\nSort entries by length descending:\n```ps\nwl sort length list.txt --descending\n```\n\nReverse sort order:\n```ps\nwl sort reverse list.txt\n```\n\n\n### Filter\nRemove whitespace entries:\n```ps\nwl filter whitespace list.txt output.txt\n```\n\nRemove duplicates\n```ps\nwl filter distinct list.txt output.txt\n```\n\nFilter entries which match (contain) a regular expression:\n```ps\nwl filter regex list.txt output.txt \\d+\n```\n\nFilter entries by a regular expression by full match:\n```ps\nwl filter regex list.txt output.txt \\A\\d+\\Z\n```\n\nFilter entries by minimum length:\n```ps\nwl filter min-length list.txt output.txt --length 3\n```\n\nFilter entries by maximum length:\n```ps\nwl filter max-length list.txt output.txt --length 3\n```\n\n\n### Transform\nTrim leading and trailing whitespace:\n```ps\nwl transform trim list.txt output.txt\n```\n\nConvert entries to uppercase:\n```ps\nwl transform uppercase list.txt output.txt\n```\n\nConvert entries to lowercase:\n```ps\nwl transform lowercase list.txt output.txt\n```\n\nPrepend to entries:\n```ps\nwl transform prepend list.txt output.txt --value \"prefix\"\n```\n\nAppend to entries:\n```ps\nwl transform append list.txt output.txt --value \"suffix\"\n```\n\nReverse entries:\n```ps\nwl transform reverse list.txt output.txt\n```\n\nNo not change entries. This transform is useful to either display:\n```ps\nwl transform identity list.txt OUT\n```\n\nOr change encoding or line endings of a list:\n```ps\nwl transform identity in.txt out.txt --input-encoding UTF8 --output-encoding ASCII\n```\n\n\n### Merge\nConcatenate multiple lists together:\n```ps\nwl merge concat --inputs list1.txt list-*.txt --output output.txt\n```\n\nConcatenate multiple lists together as raw bytes without reprocessing, separated by `--output-line-ending`. This is the fastest way to concatenate lists:\n```ps\nwl merge binary-concat --inputs list1.txt list-*.txt --output output.txt\n```\n\nUnion of multiple lists:\n```ps\nwl merge union --inputs list1.txt list-*.txt --output output.txt\n```\n\nCombine/zip together multiple lists line by line with no separator:\n```ps\nwl merge zip --inputs list1.txt list2.txt --output output.txt\n```\n\nCombine/zip together multiple lists line by line with a separator:\n```ps\nwl merge zip --inputs list1.txt list2.txt --output output.txt --separator \":\"\n```\n\nCombine each line with each other line with no separator:\n```ps\nwl merge cross --inputs list1.txt list2.txt --output output.txt\n```\n\nCombine each line with each other line with a separator:\n```ps\nwl merge cross --inputs list1.txt list2.txt --output output.txt --separator \":\"\n```\n\nCombine each line with each other line of itself with a separator:\n```ps\nwl merge cross --inputs single.txt --output output.txt --separator \":\"\n```\n\nRemove entries which can be found in other lists:\n```ps\nwl merge except --inputs list.txt except-these.txt and-these-*.txt --output output.txt\n```\n\n\n### List operations\nTake first N entries:\n```ps\nwl list take list.txt output.txt --count 500\n```\n\nTake last N entries:\n```ps\nwl list take-last list.txt output.txt --count 500\n```\n\nSkip first N entries:\n```ps\nwl list skip list.txt output.txt --count 500\n```\n\nSkip last N entries:\n```ps\nwl list skip-last list.txt output.txt --count 500\n```\n\n\n### Split\nSplit a single list into multiple chunks by number of entries:\n```ps\nwl split entries list.txt output-{0}.txt --count 500\n```\n\nSplit into chunks by number of bytes (but do not break any entries):\n```ps\nwl split bytes list.txt output-{0}.txt --bytes 1048576\n```\n\nSplit by length:\n```ps\nwl split length list.txt output-{0}.txt\n```\n\nSplit by regular expression into `matching` and `non-matching` lists:\n```ps\nwl split regex list.txt output-{0}.txt --regex \\d+\n```\n\n\n### Extract\nExtract words from files using a regular expression (`\\w+` by default):\n```ps\nwl extract regex --inputs books/*.txt --output output.txt \n```\n\nUsing a custom regular expression:\n```ps\nwl extract regex --inputs books/*.txt --output output.txt --regex [a-z]+\n```\n\n\n### Generate\nGenerate entries using a specific charset:\n```ps\nwl generate new output.txt --charset 0123456789 --min-length 1 --max-length 5\n```\n\n\n## Configuration\n\n### Input bindings cardinality\nWordlist Tool tries to process lists by streaming to avoid locking resources and buffering whole lists into memory (whenever possible).\nThis is why in general both an input and output path must be provided:\n```ps\nwl transform trim intput.txt output.txt\n```\n\nIn some special cases, where buffering is inevitable (e.g. sorting, distinct), the same path can be used:\n```ps\nwl sort asc file.txt\n```\n\nSome operations support working with multiple inputs (merge, extract). You can either specify multiple files by their path:\n```\nwl merge union --inputs file1.txt file2.txt --output out.txt\n```\n\nOr use glob patterns:\n```\nwl merge union --inputs first.txt file-*.txt last.txt --output out.txt\n```\n\n*Note: patterns are evaluated individually, and if multiple patterns match the same file, it is going to be included multiple times.*\n\n### Standard Input/Output bindings and chaining\nYou can use the reserved word `IN` and `OUT` to bind either input or output to standard input/output:\n```ps\nwl transform lower file.txt OUT\n```\n\nThis makes it possible to chain commands together:\n```ps\nwl transform lower file.txt OUT |\nwl filter distinct IN OUT |\nwl sort asc IN final.txt\n```\n\n### Encoding\nEncoding can be specified, default is `ASCII`:\n```ps\nwl transform lower in.txt out.txt --input-encoding UTF-8 --output-encoding ASCII\n```\n\nYou can use the `identity` transform to change encoding of a list:\n```ps\nwl transform identity in.txt out.txt --input-encoding UTF8 --output-encoding ASCII\n```\n\n### Line endings\nLine endings can be specified in HEX notation, default is `0A`:\n```ps\nwl transform lower in.txt out.txt --input-line-ending 0D0A --output-line-ending 0A\n```\n\nYou can use the `identity` transform to change line endings of a list:\n```ps\nwl transform identity in.txt out.txt --input-line-ending 0D0A --output-line-ending 0A\n```\n\n### Buffering\nRead and write buffering can be specified the following way:\n```ps\nwl transform lower in.txt out.txt --input-buffer-size 4096 --output-buffer-size 16384\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter-juhasz%2Fwordlist-tool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeter-juhasz%2Fwordlist-tool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeter-juhasz%2Fwordlist-tool/lists"}