Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/usethisname1419/connectioncrawler
crawls a website and checks for connections
https://github.com/usethisname1419/connectioncrawler
connection crawler http-headers reporting website-analyzer
Last synced: about 4 hours ago
JSON representation
crawls a website and checks for connections
- Host: GitHub
- URL: https://github.com/usethisname1419/connectioncrawler
- Owner: usethisname1419
- Created: 2024-03-01T17:38:44.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2024-03-01T21:54:52.000Z (11 months ago)
- Last Synced: 2024-03-02T19:41:09.735Z (11 months ago)
- Topics: connection, crawler, http-headers, reporting, website-analyzer
- Language: Python
- Homepage:
- Size: 11.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Connection Crawler
Site Checker is a Python script that checks a website for outgoing connections, resolves domain names, and generates a report of the findings. It can be used to inspect websites for potentially malicious scripts, connections, or other suspicious activity.
Can be used to find unwanted connections and malware
## Features
- Checks for outgoing connections from the specified website.
- Resolves domain names to their corresponding IP addresses.
- Generates a detailed report containing HTTP response headers, resolved IP addresses, and connections made by the website.
- Supports analysis of HTML tags including , ![]()
, , and <link>.
- Uses BeautifulSoup for HTML parsing and Requests for HTTP requests.
## Usage
1. Clone the repository:
```
git clone https://github.com/your-username/site-checker.git
```
2. Navigate to the project directory:
```
cd site-checker
```
3. Install the required dependencies:
```
pip install -r requirements.txt
```
4. Run the script:
```
python crawler.py
```
5. Follow the prompts to enter the URL to check and specify the filename for the report.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Acknowledgements
- [Requests](https://docs.python-requests.org/en/latest/) - HTTP library for making requests in Python.
- [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) - Python library for pulling data out of HTML and XML files.