Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/calc1f4r/recusive-web-crawler
https://github.com/calc1f4r/recusive-web-crawler
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/calc1f4r/recusive-web-crawler
- Owner: calc1f4r
- Created: 2023-11-01T12:03:52.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2023-11-01T12:15:33.000Z (about 1 year ago)
- Last Synced: 2023-11-01T13:28:53.475Z (about 1 year ago)
- Language: Python
- Size: 2.93 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
### Recursive Web Crawler
Recursive Web Crawler is a Python-based tool for exploring websites recursively and extracting useful information such as subdomains, links, and JavaScript files. This tool is intended for web security professionals and web developers who want to examine the structure and dependencies of websites.
https://github.com/calc1f4r/Recusive-web-crawler/assets/74751675/bfd97074-2aab-44c9-8b6d-33f7f72c71df
#### Features
- Recursively crawl websites to a specified depth.
- Extract subdomains, links, and JavaScript files.
- Flexible depth parameter for customizing the level of recursion.
##### Usage
Run the Recursive Web Crawler with the following command:
```python
python main.py -u -d
```
Example
```python
python main.py -u "'https://tryhackme.com -d 2
```
##### Contributing
If you'd like to contribute to this project, please open an issue or create a pull request.