https://github.com/srijanmukherjee/dataharvestors
Various web scrapers for collecting data
https://github.com/srijanmukherjee/dataharvestors
Last synced: 3 months ago
JSON representation
Various web scrapers for collecting data
- Host: GitHub
- URL: https://github.com/srijanmukherjee/dataharvestors
- Owner: srijanmukherjee
- License: mit
- Created: 2023-02-28T10:45:27.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-02-28T11:33:47.000Z (over 2 years ago)
- Last Synced: 2025-01-06T01:25:35.187Z (5 months ago)
- Language: Python
- Size: 1.95 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DataHarvesters
Various web scrapers for collecting data
## How to run
Create a virtual environment
```console
python -m venv env
```
[Activate](https://docs.python.org/3/library/venv.html#how-venvs-work) the environemnt and install the dependencies
```console
pip install -r requirements.txt
```
### goodreads
```console
scrapy runspider goodreads/BookSpider.py -O output/books.json
```