https://github.com/sedrubal/webcrawler

Crawl sites and search for security issues.
https://github.com/sedrubal/webcrawler

crawler script security website-auditing

Last synced: 1 day ago
JSON representation

Crawl sites and search for security issues.

• Host: GitHub
• URL: https://github.com/sedrubal/webcrawler
• Owner: sedrubal
• Created: 2017-07-18T11:52:16.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2017-10-07T19:44:11.000Z (over 7 years ago)
• Last Synced: 2024-11-24T05:11:54.551Z (2 months ago)
• Topics: crawler, script, security, website-auditing
• Language: Python
• Size: 14.6 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# WebCrawler

Crawl sites and search for security issues.

## Installation

- `git clone [email protected]:sedrubal/WebCrawler.git && cd ./WebCrawler/`

- `python3 -m virtualenv -p python3.6 --system-site-packages .venv && . ./.venv/bin/activate`

- `pip3 install -r ./requirements.txt`

- `./webcrawler.py -vvv ./config-example.yml -`

## Usage

```

usage: webcrawler.py [-h] [-v] config_file out_file

Crawl all configured sites and search for security issues.

positional arguments:

  config_file    The yaml config file

  out_file       The yaml file to write the output

optional arguments:

  -h, --help     show this help message and exit

  -v, --verbose  More output

```

## Idea

- https://www.golem.de/news/https-private-schluessel-auf-dem-webserver-1707-128860.html

- https://www.golem.de/news/kundendaten-datenleck-bei-der-deutschen-post-1707-128751.html

- https://www.golem.de/news/sicherheitsluecke-fehlerhaft-konfiguriertes-git-verzeichnis-bei-redcoon-1705-127777.html

## License

[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)