https://github.com/calc1f4r/recusive-web-crawler
"Recursive Web Crawler: A Python tool for deep website exploration, finding subdomains, links, and JavaScript files. Ideal for security and web development."
https://github.com/calc1f4r/recusive-web-crawler
crawling multithreaded offensive python
Last synced: 5 months ago
JSON representation
"Recursive Web Crawler: A Python tool for deep website exploration, finding subdomains, links, and JavaScript files. Ideal for security and web development."
- Host: GitHub
- URL: https://github.com/calc1f4r/recusive-web-crawler
- Owner: calc1f4r
- Created: 2023-11-01T12:03:52.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2023-11-01T12:15:33.000Z (over 1 year ago)
- Last Synced: 2024-11-14T18:39:12.434Z (7 months ago)
- Topics: crawling, multithreaded, offensive, python
- Language: Python
- Homepage:
- Size: 2.93 KB
- Stars: 5
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
### Recursive Web Crawler
Recursive Web Crawler is a Python-based tool for exploring websites recursively and extracting useful information such as subdomains, links, and JavaScript files. This tool is intended for web security professionals and web developers who want to examine the structure and dependencies of websites.
https://github.com/calc1f4r/Recusive-web-crawler/assets/74751675/bfd97074-2aab-44c9-8b6d-33f7f72c71df
#### Features
- Recursively crawl websites to a specified depth.
- Extract subdomains, links, and JavaScript files.
- Flexible depth parameter for customizing the level of recursion.
##### Usage
Run the Recursive Web Crawler with the following command:
```python
python main.py -u -d
```
Example
```python
python main.py -u "'https://tryhackme.com -d 2
```
##### Contributing
If you'd like to contribute to this project, please open an issue or create a pull request.