https://github.com/xprnvd/makdi
Website crawler created for pentest exercises like HTB.
https://github.com/xprnvd/makdi
crawler htb htb-scripts pentest python
Last synced: 8 months ago
JSON representation
Website crawler created for pentest exercises like HTB.
- Host: GitHub
- URL: https://github.com/xprnvd/makdi
- Owner: xprnvd
- Created: 2021-01-17T16:30:50.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2022-05-22T18:59:13.000Z (almost 4 years ago)
- Last Synced: 2024-01-17T20:19:22.855Z (about 2 years ago)
- Topics: crawler, htb, htb-scripts, pentest, python
- Language: Python
- Homepage:
- Size: 17.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## makdi
Website crawler created for pentest exercises like HTB.
The 2.x community versions of Burp Proxy don't include an active spider tool. This tools is designed to crawl only the internal resources found through html < a > tags. After the links have been crawled, it will send the unique values through the proxy. The execution will end, if over a certain duration ,the internal link set has not increased or it hits the defined hard stop value.
As the links are only printed to stdout, if proxy is inaccessible, here a recording where the proxy connection has failed. Export funtion can be uncommented, if you'd like the links to be printed regardless of the proxy connection state.
[](https://asciinema.org/a/RNgdvIlycQdPQzqIiTzJ7XQK7)