https://github.com/springload/site-url-scraper
Multithreaded Python application to search a string pattern or find all internal links from a url or CSV list of urls
https://github.com/springload/site-url-scraper
python scraper
Last synced: 4 months ago
JSON representation
Multithreaded Python application to search a string pattern or find all internal links from a url or CSV list of urls
- Host: GitHub
- URL: https://github.com/springload/site-url-scraper
- Owner: springload
- Created: 2015-01-26T03:43:37.000Z (about 11 years ago)
- Default Branch: master
- Last Pushed: 2017-09-04T00:19:13.000Z (over 8 years ago)
- Last Synced: 2025-10-09T19:05:03.370Z (4 months ago)
- Topics: python, scraper
- Language: Python
- Homepage:
- Size: 29.3 KB
- Stars: 3
- Watchers: 25
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Site url scraper
Multithreaded Python application to search a string pattern or find all internal links from a url or CSV list of urls
**/!\ This project isn't actively maintained. Check alternatives like [pylinkchecker](https://github.com/mtlevolio/pylinkchecker) or [broken-link-checker](https://github.com/stevenvachon/broken-link-checker) /!\\**
## Installation
```bash
git clone REPOSITORY_URL
cd site-url-scraper
pip install -e .
```
## Usage
```bash
python GLSapp/Main.py --help
```