{"id":21443894,"url":"https://github.com/scalabli/webprobe","last_synced_at":"2025-09-08T00:13:50.034Z","repository":{"id":62592840,"uuid":"512687489","full_name":"scalabli/webprobe","owner":"scalabli","description":"Nifty and sophisticated web path scanner","archived":false,"fork":false,"pushed_at":"2022-09-18T19:59:47.000Z","size":986,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-17T01:22:47.580Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scalabli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"scalabli","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null}},"created_at":"2022-07-11T09:09:02.000Z","updated_at":"2022-11-08T19:27:04.000Z","dependencies_parsed_at":"2022-11-03T23:05:20.545Z","dependency_job_id":null,"html_url":"https://github.com/scalabli/webprobe","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/scalabli/webprobe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scalabli%2Fwebprobe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scalabli%2Fwebprobe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scalabli%2Fwebprobe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scalabli%2Fwebprobe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scalabli","download_url":"https://codeload.github.com/scalabli/webprobe/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scalabli%2Fwebprobe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274113620,"owners_count":25224447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T02:14:45.579Z","updated_at":"2025-09-08T00:13:50.016Z","avatar_url":"https://github.com/scalabli.png","language":"Python","readme":"[![Downloads](https://pepy.tech/badge/webprobe)](https://pepy.tech/project/webprobe)\n[![PyPI version](https://badge.fury.io/py/webprobe.svg)](https://badge.fury.io/py/webprobe)\n[![Wheel](https://img.shields.io/pypi/wheel/webprobe.svg)](https://pypi.com/project/webprobe)\n[![licence](https://img.shields.io/pypi/l/webprobe.svg)](https://opensource.org/licenses/MIT)\n[![Twitter Follow](https://img.shields.io/twitter/follow/gerrishon_s.svg?style=social)](https://twitter.com/gerrishon_s)\n\n[![Logo](https://raw.githubusercontent.com/scalabli/webprobe/main/images/webprobe.png)](https://github.com/scalabli/webprobe)\n\n# webprobe\nNifty and sophisticated web path scanner for security researchers\n\nA sophisticated web path scanner designed for the most descerning bug bounty hunters.\n\n\nThis CLI brute forces directories and files in webservers.\n\n\n### Installation \u0026 Usage\n                                                \n**Requirement: python 3.8 or higher**\n\n- Install with PyPi: `pip install webprobe`\n\n\n### How to use\n\nSome common examples on how to use webprobe.\n\n:bulb: If you need to see a list of all options, just use the `-h | --help` argument.\n\nTo use multiple wordlists, you can separate your wordlists with commas. Example: `wordlist1.txt,wordlist2.txt`\n\n### Simple usage\n\n```python\n\nwebprobe -u https://example.com\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com\n\n```\n\n```python\nwebprobe -e php,html,js -u https://exaple.com -w /path/to/wordlist\n```\n\n## Pausing progress\n\nYou can pause the scanning progress with CTRL+C  from here, you can save the progress (and continue later), skip the current target, or skip the current sub-directory.\n\n## Recursion\n- Brutforcing recursively can be achieved using `-r` or `--recursive` flag.\n\nFor example, if webprobe finds `admin/`, it will brute-force `admin/* ` (`*` is where it brute forces).\n\n```python\n\nwebprobe -e php,html,js -u https://example.com -r\n\n```\n\n- You can set the max recursion depth with `--recursion-depth` and status codes to recurse with `--recursion-status`\n\n```python\n\nwebprobe -e php,html,js -u https://example.com -r --recursion-depth 3 --recursion-status 200-39\n\n```\n- You can brute force recursively all found paths, not just paths end with `/` using `--force-recursive` flag.\n\n- You can recursively brute-force all depths of a path (`a/b/c` =\u003e add `a/`, `a/b/`) using `--deep-recursive` flag.\n\n- If there are sub-directories that you do not want to brute-force recursively use `--exclude-subdirs` flag.\n\n```python\n\nwebprobe -e php,html,js -u https://example.com -r --exclude-subdirs image/,media/,css/\n\n```\n\n## Threads\nThread number (`-t | --threads`) reflects the number of separated brute force processes. The bigger the thread number, the faster webprobe runs. By default, the number of threads is 25, but you can increase it if you want to speed up the progress.\n\nHowever, the speed still depends on the response time of the server.\n\n:bulb: keep the threads number within a reasonable range because it can cause DoS (Denial of Service).\n\n```python\n\nwebprobe -e php,htm,js,bak,zip,tgz,txt -u https://example.com -t 20\n\n```\n\n## Filters\nUse **-i | --include-status** and **-x | --exclude-status** flags to select allowed and not allowed response status-codes\n\nFor more advanced filters: **--exclude-sizes**, **--exclude-texts**, **--exclude-regexps**, **--exclude-redirects** and **--exclude-response**\n\n```python\n\nwebprobe  -e php,html,js -u https://example.com --exclude-sizes 1B,243KB\n\n```                                                      \n```python\n\nwebprobe -e php,html,js -u https://example.com --exclude-texts \"403 Forbidden\"\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --exclude-regexps \"^Error$\"\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --exclude-redirects \"https://(.*).okta.com/*\"\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --exclude-response /error.html\n\n```\n\n## Scan sub-directories\n- You can scan a list of sub-directories with **--subdirs** flag.\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --subdirs /,admin/,folder/\n```\n## Proxies\n- Webprobe supports both SOCKS and HTTP proxy. You can enlist a proxy server or a list of proxy servers from a file.\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --proxy 127.0.0.1:8080\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --proxy socks5://10.10.0.1:8080\n\n```\n\n```python\n\nwebprobe -e php,html,js -u https://example.com --proxylist proxyservers.txt\n\n```\n## More example commands\n\n```python\n\ncat urls.txt | python3 webprobe --stdin\n\n```\n\n```python\n\nwebprobe -u https://example.com --max-time 360\n\n```\n\n```python\n\nwebprobe -u https://example.com --auth admin:pass --auth-type basic\n\n```\n\n```python\n\nwebprobe -u https://example.com --header-list rate-limit-bypasses.txt\n\n```\n\n## Reports\nSupported report formats are: **simple**, **plain**, **json**, **xml**, **md**, **csv**,  **html**, **sqlite**\n\n:bulb: We will be adding `yaml` soon\n\n```python\n webprobe -e php -l URLs.txt --format plain -o report.txt\n```\n\n```python\n webprobe -e php -u https://example.com --format html -o example.json\n```\n\n## License📑\n\n [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nThis software is licensed under the `MIT License`. See the [License](https://github.com/scalabli/webprobe/blob/master/LICENSE) file in the top distribution directory for the full license text.\n\n## Donate🎁\nIn order to for us to maintain this project and grow our community of contributors.\n[Donate](https://ko-fi.com/scalabli)\n\n\n## Code of Conduct\nCode of Conduct is adapted from the Contributor Covenant, version 1.2.0 available at [Code of Conduct](http://contributor-covenant.org/version/1/2/0/)\n","funding_links":["https://ko-fi.com/scalabli"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscalabli%2Fwebprobe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscalabli%2Fwebprobe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscalabli%2Fwebprobe/lists"}