{"id":13815259,"url":"https://github.com/cytopia/urlbuster","last_synced_at":"2025-04-09T18:23:08.866Z","repository":{"id":41494591,"uuid":"236336753","full_name":"cytopia/urlbuster","owner":"cytopia","description":"Powerful mutable web directory fuzzer to bruteforce existing and/or hidden files or directories.","archived":false,"fork":false,"pushed_at":"2021-01-30T13:37:57.000Z","size":100,"stargazers_count":161,"open_issues_count":2,"forks_count":29,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-02T10:44:32.692Z","etag":null,"topics":["brute-force","bruteforce","bruteforce-attacks","crawler","cytopia-sec","url-bruteforcer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cytopia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-26T16:05:06.000Z","updated_at":"2025-03-03T02:27:05.000Z","dependencies_parsed_at":"2022-07-12T18:18:11.722Z","dependency_job_id":null,"html_url":"https://github.com/cytopia/urlbuster","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cytopia%2Furlbuster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cytopia%2Furlbuster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cytopia%2Furlbuster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cytopia%2Furlbuster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cytopia","download_url":"https://codeload.github.com/cytopia/urlbuster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248085940,"owners_count":21045243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["brute-force","bruteforce","bruteforce-attacks","crawler","cytopia-sec","url-bruteforcer"],"created_at":"2024-08-04T04:03:13.412Z","updated_at":"2025-04-09T18:23:08.845Z","avatar_url":"https://github.com/cytopia.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"```\n   ██╗   ██╗██████╗ ██╗     ██████╗ ██╗   ██╗███████╗████████╗███████╗██████╗\n   ██║   ██║██╔══██╗██║     ██╔══██╗██║   ██║██╔════╝╚══██╔══╝██╔════╝██╔══██╗\n   ██║   ██║██████╔╝██║     ██████╔╝██║   ██║███████╗   ██║   █████╗  ██████╔╝\n   ██║   ██║██╔══██╗██║     ██╔══██╗██║   ██║╚════██║   ██║   ██╔══╝  ██╔══██╗\n   ╚██████╔╝██║  ██║███████╗██████╔╝╚██████╔╝███████║   ██║   ███████╗██║  ██║\n    ╚═════╝ ╚═╝  ╚═╝╚══════╝╚═════╝  ╚═════╝ ╚══════╝   ╚═╝   ╚══════╝╚═╝  ╚═╝\n```\n\n[![](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![PyPI](https://img.shields.io/pypi/v/urlbuster)](https://pypi.org/project/urlbuster/)\n[![PyPI - Status](https://img.shields.io/pypi/status/urlbuster)](https://pypi.org/project/urlbuster/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/urlbuster)](https://pypi.org/project/urlbuster/)\n[![PyPI - Format](https://img.shields.io/pypi/format/urlbuster)](https://pypi.org/project/urlbuster/)\n[![PyPI - Implementation](https://img.shields.io/pypi/implementation/urlbuster)](https://pypi.org/project/urlbuster/)\n[![PyPI - License](https://img.shields.io/pypi/l/urlbuster)](https://pypi.org/project/urlbuster/)\n\n[![Build Status](https://github.com/cytopia/urlbuster/workflows/linting/badge.svg)](https://github.com/cytopia/urlbuster/actions?workflow=linting)\n[![Build Status](https://github.com/cytopia/urlbuster/workflows/building/badge.svg)](https://github.com/cytopia/urlbuster/actions?workflow=building)\n[![Build Status](https://github.com/cytopia/urlbuster/workflows/testing/badge.svg)](https://github.com/cytopia/urlbuster/actions?workflow=testing)\n\n\nPowerful web directory fuzzer to locate existing and/or hidden files or directories.\n\nSimilar to [dirb](http://dirb.sourceforge.net/) or [gobuster](https://github.com/OJ/gobuster), but\nwith a lot of mutation options.\n\n\n## :tada: Installation\n```bash\npip install urlbuster\n```\n\n\n## :star: Features\n\n* Proxy support\n* Cookie support\n* Basic Auth\n* Digest Auth\n* Retries (for slow servers)\n* Persistent and non-persistent HTTP connection\n* Request methods: GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS\n* Custom HTTP header\n* Mutate POST, PUT and PATCH payloads\n* Mutate with different request methods\n* Mutate with different HTTP headers\n* Mutate with different file extensions\n* Mutate with and without trailing slashes\n* Enumerate GET parameter values\n\n\n## :computer: Usage\n```\nusage: urlbuster [options] -w \u003cstr\u003e/-W \u003cfile\u003e BASE_URL\n       urlbuster -V, --help\n       urlbuster -h, --version\n\nURL bruteforcer to locate existing and/or hidden files or directories.\n\nSimilar to dirb or gobuster, but also allows to iterate over multiple HTTP request methods,\nmultiple useragents and multiple host header values.\n\npositional arguments:\n  BASE_URL              The base URL to scan.\n\nrequired arguments:\n  -w str, --word str    Word to use.\n  -W f, --wordlist f    Path to wordlist to use.\n\noptional global arguments:\n  -n, --new             Use a new connection for every request.\n                        If not specified persistent http connection will be used for all requests.\n                        Note, using a new connection will decrease performance,\n                        but ensure to have a clean state on every request.\n                        A persistent connection on the other hand will use any additional cookie values\n                        it has received from a previous request.\n  -f, --follow          Follow redirects.\n  -k, --insecure        Do not verify TLS certificates.\n  -v, --verbose         Show also missed URLs.\n  --code str [str ...]  HTTP status code to treat as success.\n                        You can use a '.' (dot) as a wildcard.\n                        Default: 2.. 3.. 403 407 411 426 429 500 505 511\n  --payload p [p ...]   POST, PUT and PATCH payloads for all requests.\n                        Note, multiple values are allowed for multiple payloads.\n                        Note, if duplicates are specified, the last one will overwrite.\n                        See --mpayload for mutations.\n                        Format: \u003ckey\u003e=\u003cval\u003e [\u003ckey\u003e=\u003cval\u003e]\n  --header h [h ...]    Custom http header string to add to all requests.\n                        Note, multiple values are allowed for multiple headers.\n                        Note, if duplicates are specified, the last one will overwrite.\n                        See --mheaders for mutations.\n                        Format: \u003ckey\u003e:\u003cval\u003e [\u003ckey\u003e:\u003cval\u003e]\n  --cookie c [c ...]    Cookie string to add to all requests.\n                        Format: \u003ckey\u003e=\u003cval\u003e [\u003ckey\u003e=\u003cval\u003e]\n  --proxy str           Use a proxy for all requests.\n                        Format: http://\u003chost\u003e:\u003cport\u003e\n                        Format: http://\u003cuser\u003e:\u003cpass\u003e@\u003chost\u003e:\u003cport\u003e\n                        Format: https://\u003chost\u003e:\u003cport\u003e\n                        Format: https://\u003cuser\u003e:\u003cpass\u003e@\u003chost\u003e:\u003cport\u003e\n                        Format: socks5://\u003chost\u003e:\u003cport\u003e\n                        Format: socks5://\u003cuser\u003e:\u003cpass\u003e@\u003chost\u003e:\u003cport\u003e\n  --auth-basic str      Use basic authentication for all requests.\n                        Format: \u003cuser\u003e:\u003cpass\u003e\n  --auth-digest str     Use digest authentication for all requests.\n                        Format: \u003cuser\u003e:\u003cpass\u003e\n  --timeout sec         Connection timeout in seconds for each request.\n                        Default: 5.0\n  --retry num           Connection retries per request.\n                        Default: 3\n  --delay sec           Delay between requests to not flood the server.\n  --output file         Output file to write results to.\n\noptional mutating arguments:\n  The following arguments will increase the total number of requests to be made by\n  applying various mutations and testing each mutation on a separate request.\n\n  --method m [m ...]    List of HTTP methods to test each request against.\n                        Note, each supplied method will double the number of requests.\n                        Supported methods: GET POST PUT DELETE PATCH HEAD OPTIONS\n                        Default: GET\n  --mpayload p [p ...]  POST, PUT and PATCH payloads to mutate all requests..\n                        Note, multiple values are allowed for multiple payloads.\n                        Format: \u003ckey\u003e=\u003cval\u003e [\u003ckey\u003e=\u003cval\u003e]\n  --mheader h [h ...]   Custom http header string to add to mutate all requests.\n                        Note, multiple values are allowed for multiple headers.\n                        Format: \u003ckey\u003e:\u003cval\u003e [\u003ckey\u003e:\u003cval\u003e]\n  --ext ext [ext ...]   List of file extensions to to add to words for testing.\n                        Note, each supplied extension will double the number of requests.\n                        Format: .zip [.pem]\n  --slash str           Append or omit a trailing slash to URLs to test.\n                        Note, a slash will be added after the extensions if they are specified as well.\n                        Note, using 'both' will double the number of requests.\n                        Options: both, yes, no\n                        Default: no\n\nmisc arguments:\n  -h, --help            Show this help message and exit\n  -V, --version         Show version information\n\nexamples\n\n  urlbuster -W /path/to/words http://example.com/\n  urlbuster -W /path/to/words http://example.com:8000/\n  urlbuster -k -W /path/to/words https://example.com:10000/\n```\n\n\n## :bulb: Mutation example\n\nSome websites behave differently for the same path depending on the specified useragent.\n\n```bash\n$ urlbuster \\\n  -W /usr/share/dirb/wordlists/common.txt \\\n  --mheader 'User-Agent:Googlebot/2.1 (+http://www.googlebot.com/bot.html)' \\\n  --method 'POST,GET,DELETE,PUT,PATCH' \\\n  http://www.domain.tld/\n```\n\n```\n   ██╗   ██╗██████╗ ██╗     ██████╗ ██╗   ██╗███████╗████████╗███████╗██████╗\n   ██║   ██║██╔══██╗██║     ██╔══██╗██║   ██║██╔════╝╚══██╔══╝██╔════╝██╔══██╗\n   ██║   ██║██████╔╝██║     ██████╔╝██║   ██║███████╗   ██║   █████╗  ██████╔╝\n   ██║   ██║██╔══██╗██║     ██╔══██╗██║   ██║╚════██║   ██║   ██╔══╝  ██╔══██╗\n   ╚██████╔╝██║  ██║███████╗██████╔╝╚██████╔╝███████║   ██║   ███████╗██║  ██║\n    ╚═════╝ ╚═╝  ╚═╝╚══════╝╚═════╝  ╚═════╝ ╚══════╝   ╚═╝   ╚══════╝╚═╝  ╚═╝\n\n                               0.5.1 by cytopia\n\n      SETTINGS\n            Base URL:          https://www.everythingcli.org/\n            Valid codes:       2.., 3.., 403, 407, 411, 426, 429, 500, 505, 511\n            Connection:        Non-persistent\n            Redirects:         Don't follow\n            Payloads:          None\n            Timeout:           5.0s\n            Retries:           3\n            Delay:             None\n\n      MUTATIONS\n            Mutating headers:  2\n            Mutating payloads: 0 (POST)\n            Methods:           5 (POST, GET, DELETE, PUT, PATCH)\n            Slashes:           no\n            Extensions:        1 (empty extension)\n            Words:             4614\n\n      TOTAL REQUESTS: 46140\n      START TIME:     2020-01-29 08:52:12\n\n\n--------------------------------------------------------------------------------\nConnection:      keep-alive\nAccept-Encoding: gzip, deflate\nAccept:          */*\nUser-Agent:      python-requests/2.22.0\n\n[301] [GET]      http://domain.tld/robots.txt\n\n--------------------------------------------------------------------------------\nConnection:      keep-alive\nAccept-Encoding: gzip, deflate\nAccept:          */*\nUser-Agent:      Googlebot/2.1 (+http://www.googlebot.com/bot.html)\n\n[200] [GET]      http://domain.tld/robots.txt\n[301] [POST]     http://domain.tld/robots.txt\n[301] [GET]      http://domain.tld/robots.txt\n[301] [DELETE]   http://domain.tld/robots.txt\n[301] [PUT]      http://domain.tld/robots.txt\n[301] [PATCH]    http://domain.tld/robots.txt\n```\n\n\n## :bulb: Examples\n\n### Default usage\n\n#### Basic\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  http://www.domain.tld/\n```\n#### Proxy through Burpsuite\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --proxy 'http://localhost:8080' \\\n  http://www.domain.tld/\n```\n#### Save results to file\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --output out.txt \\\n  http://www.domain.tld/\n```\n#### Scan behind Basic Auth\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --auth-basic 'user:pass' \\\n  http://www.domain.tld/\n```\n#### Use session cookie\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --cookie 'PHPSESSID=a79b00e7-035a-2bb4-352a-439d855feabf' \\\n  http://www.domain.tld/\n```\n\n\n### Find files\n\n#### Find files in root directory\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --code 200 301 302 \\\n  --ext .zip .tar .tar.gz .gz .rar \\\n  http://www.domain.tld/\n```\n#### Find files in sub directory\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --code 200 301 302 \\\n  --ext .zip .tar .tar.gz .gz .rar \\\n  http://www.domain.tld/wp-content/\n```\n\n\n### Advanced usage\n\n#### Bruteforce query parameter\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --method GET \\\n  --code 200 301 302 \\\n  http://www.domain.tld/search?q=\n```\n#### Bruteforce POST requests\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --code 200 301 302 \\\n  --method POST \\\n  --payload \\\n    'user=somename' \\\n    'pass=somepass' \\\n    'mail=some@mail.tld' \\\n    'submit=yes' \\\n  http://www.domain.tld/\n```\n#### Bruteforce mutated POST requests\n```bash\n$ urlbuster \\\n  -w index.php \\\n  --code 200 301 302 \\\n  --method POST \\\n  --mpayload \\\n    'user=somename1' \\\n    'user=somename2' \\\n    'user=somename3' \\\n    'pass=somepass1' \\\n    'pass=somepass2' \\\n    'pass=somepass3' \\\n    'mail=some@mail1.tld' \\\n    'mail=some@mail2.tld' \\\n    'mail=some@mail3.tld' \\\n    'submit=yes' \\\n  http://www.domain.tld/wp-admin/\n```\n#### Useragent SQL injections\n```bash\n$ urlbuster \\\n  -W /path/to/wordlist.txt \\\n  --code 5.. \\\n  --method GET POST \\\n  --mheader \\\n    \"User-Agent: ;\" \\\n    \"User-Agent: ' or \\\"\" \\\n    \"User-Agent: -- or #\" \\\n    \"User-Agent: ' OR '1\" \\\n    \"User-Agent: ' OR 1 -- -\" \\\n    \"User-Agent: \\\" OR 1 = 1 -- -\" \\\n    \"User-Agent: '='\" \\\n    \"User-Agent: 'LIKE'\" \\\n    \"User-Agent: '=0--+\" \\\n    \"User-Agent:  OR 1=1\" \\\n    \"User-Agent: ' OR 'x'='x\" \\\n    \"User-Agent: ' AND id IS NULL; --\" \\\n  http://www.domain.tld/\n```\n#### Find potential vhosts\n```bash\n$ urlbuster \\\n  -w / \\\n  --method GET POST \\\n  --mheader \\\n    \"Host: internal1.lan\" \\\n    \"Host: internal2.lan\" \\\n    \"Host: internal3.lan\" \\\n    \"Host: internal4.lan\" \\\n    \"Host: internal5.lan\" \\\n    \"Host: internal6.lan\" \\\n  http://10.0.0.1\n```\n\n\n## :lock: [cytopia](https://github.com/cytopia) sec tools\n\nBelow is a list of sec tools and docs I am maintaining.\n\n| Name                 | Category             | Language   | Description |\n|----------------------|----------------------|------------|-------------|\n| **[offsec]**         | Documentation        | Markdown   | Offsec checklist, tools and examples |\n| **[header-fuzz]**    | Enumeration          | Bash       | Fuzz HTTP headers |\n| **[smtp-user-enum]** | Enumeration          | Python 2+3 | SMTP users enumerator |\n| **[urlbuster]**      | Enumeration          | Python 2+3 | Mutable web directory fuzzer |\n| **[netcat]**         | Pivoting             | Python 2+3 | Cross-platform netcat |\n| **[badchars]**       | Reverse Engineering  | Python 2+3 | Badchar generator |\n| **[fuzza]**          | Reverse Engineering  | Python 2+3 | TCP fuzzing tool |\n\n[offsec]: https://github.com/cytopia/offsec\n[header-fuzz]: https://github.com/cytopia/header-fuzz\n[smtp-user-enum]: https://github.com/cytopia/smtp-user-enum\n[urlbuster]: https://github.com/cytopia/urlbuster\n[netcat]: https://github.com/cytopia/netcat\n[badchars]: https://github.com/cytopia/badchars\n[fuzza]: https://github.com/cytopia/fuzza\n\n\n## :octocat: Contributing\n\nSee **[Contributing guidelines](CONTRIBUTING.md)** to help to improve this project.\n\n\n## :exclamation: Disclaimer\n\nThis tool may be used for legal purposes only. Users take full responsibility for any actions performed using this tool. The author accepts no liability for damage caused by this tool. If these terms are not acceptable to you, then do not use this tool.\n\n\n## :page_facing_up: License\n\n**[MIT License](LICENSE.txt)**\n\nCopyright (c) 2020 **[cytopia](https://github.com/cytopia)**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcytopia%2Furlbuster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcytopia%2Furlbuster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcytopia%2Furlbuster/lists"}