Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/scalabli/webprobe
Nifty and sophisticated web path scanner
https://github.com/scalabli/webprobe
Last synced: about 1 month ago
JSON representation
Nifty and sophisticated web path scanner
- Host: GitHub
- URL: https://github.com/scalabli/webprobe
- Owner: scalabli
- License: mit
- Created: 2022-07-11T09:09:02.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-09-18T19:59:47.000Z (over 2 years ago)
- Last Synced: 2024-11-16T02:08:59.842Z (about 2 months ago)
- Language: Python
- Size: 963 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
[](https://pepy.tech/project/webprobe)
[](https://badge.fury.io/py/webprobe)
[](https://pypi.com/project/webprobe)
[](https://opensource.org/licenses/MIT)
[](https://twitter.com/gerrishon_s)
[](https://github.com/scalabli/webprobe)
# webprobe
Nifty and sophisticated web path scanner for security researchers
A sophisticated web path scanner designed for the most descerning bug bounty hunters.
This CLI brute forces directories and files in webservers.
### Installation & Usage
**Requirement: python 3.8 or higher**
- Install with PyPi: `pip install webprobe`
### How to use
Some common examples on how to use webprobe.
:bulb: If you need to see a list of all options, just use the `-h | --help` argument.
To use multiple wordlists, you can separate your wordlists with commas. Example: `wordlist1.txt,wordlist2.txt`
### Simple usage
```python
webprobe -u https://example.com
```
```python
webprobe -e php,html,js -u https://example.com
```
```python
webprobe -e php,html,js -u https://exaple.com -w /path/to/wordlist
```
## Pausing progress
You can pause the scanning progress with CTRL+C from here, you can save the progress (and continue later), skip the current target, or skip the current sub-directory.
## Recursion
- Brutforcing recursively can be achieved using `-r` or `--recursive` flag.
For example, if webprobe finds `admin/`, it will brute-force `admin/* ` (`*` is where it brute forces).
```python
webprobe -e php,html,js -u https://example.com -r
```
- You can set the max recursion depth with `--recursion-depth` and status codes to recurse with `--recursion-status`
```python
webprobe -e php,html,js -u https://example.com -r --recursion-depth 3 --recursion-status 200-39
```
- You can brute force recursively all found paths, not just paths end with `/` using `--force-recursive` flag.
- You can recursively brute-force all depths of a path (`a/b/c` => add `a/`, `a/b/`) using `--deep-recursive` flag.
- If there are sub-directories that you do not want to brute-force recursively use `--exclude-subdirs` flag.
```python
webprobe -e php,html,js -u https://example.com -r --exclude-subdirs image/,media/,css/
```
## Threads
Thread number (`-t | --threads`) reflects the number of separated brute force processes. The bigger the thread number, the faster webprobe runs. By default, the number of threads is 25, but you can increase it if you want to speed up the progress.
However, the speed still depends on the response time of the server.
:bulb: keep the threads number within a reasonable range because it can cause DoS (Denial of Service).
```python
webprobe -e php,htm,js,bak,zip,tgz,txt -u https://example.com -t 20
```
## Filters
Use **-i | --include-status** and **-x | --exclude-status** flags to select allowed and not allowed response status-codes
For more advanced filters: **--exclude-sizes**, **--exclude-texts**, **--exclude-regexps**, **--exclude-redirects** and **--exclude-response**
```python
webprobe -e php,html,js -u https://example.com --exclude-sizes 1B,243KB
```
```python
webprobe -e php,html,js -u https://example.com --exclude-texts "403 Forbidden"
```
```python
webprobe -e php,html,js -u https://example.com --exclude-regexps "^Error$"
```
```python
webprobe -e php,html,js -u https://example.com --exclude-redirects "https://(.*).okta.com/*"
```
```python
webprobe -e php,html,js -u https://example.com --exclude-response /error.html
```
## Scan sub-directories
- You can scan a list of sub-directories with **--subdirs** flag.
```python
webprobe -e php,html,js -u https://example.com --subdirs /,admin/,folder/
```
## Proxies
- Webprobe supports both SOCKS and HTTP proxy. You can enlist a proxy server or a list of proxy servers from a file.
```python
webprobe -e php,html,js -u https://example.com --proxy 127.0.0.1:8080
```
```python
webprobe -e php,html,js -u https://example.com --proxy socks5://10.10.0.1:8080
```
```python
webprobe -e php,html,js -u https://example.com --proxylist proxyservers.txt
```
## More example commands
```python
cat urls.txt | python3 webprobe --stdin
```
```python
webprobe -u https://example.com --max-time 360
```
```python
webprobe -u https://example.com --auth admin:pass --auth-type basic
```
```python
webprobe -u https://example.com --header-list rate-limit-bypasses.txt
```
## Reports
Supported report formats are: **simple**, **plain**, **json**, **xml**, **md**, **csv**, **html**, **sqlite**
:bulb: We will be adding `yaml` soon
```python
webprobe -e php -l URLs.txt --format plain -o report.txt
```
```python
webprobe -e php -u https://example.com --format html -o example.json
```
## License📑
[](https://opensource.org/licenses/MIT)
This software is licensed under the `MIT License`. See the [License](https://github.com/scalabli/webprobe/blob/master/LICENSE) file in the top distribution directory for the full license text.
## Donate🎁
In order to for us to maintain this project and grow our community of contributors.
[Donate](https://ko-fi.com/scalabli)
## Code of Conduct
Code of Conduct is adapted from the Contributor Covenant, version 1.2.0 available at [Code of Conduct](http://contributor-covenant.org/version/1/2/0/)