https://github.com/yashrsharma44/almanacscraper

Scraper that scrapes weather data from almanac.com
https://github.com/yashrsharma44/almanacscraper

Last synced: about 1 month ago
JSON representation

Scraper that scrapes weather data from almanac.com

• Host: GitHub
• URL: https://github.com/yashrsharma44/almanacscraper
• Owner: yashrsharma44
• Created: 2018-10-14T16:36:13.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-10-16T06:16:40.000Z (over 6 years ago)
• Last Synced: 2024-10-30T02:48:34.126Z (3 months ago)
• Language: Python
• Size: 19.5 KB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# AlmanacScraper

Scraper that scrapes weather data from almanac.com

## Steps to run the script

* Clone the repository using `git clone https://github.com/yashrsharma44/AlmanacScraper.git`

* Create a virtualenv `virtualenv --python=python3.5 venv`

* Type `. venv/bin/activate` to activate the virtualenv

* Install the dependencies `pip install requests` `pip install bs4` `pip install tqdm`.

* cd into the folder

* Run `python main.py`

* After use, deactivate the virtualenv using `deactivate`.

All list of data will be created, containing csv files of each states. By default, the starting and ending dates are 1st January 1998, and 2nd January 1998.

## Note 

For Parallel Processing, please use `sidpr.py`, `tapaaspr.py` and `biswapr.py`. Please set the value of PROCS, to reasonable number as you consider the value according to specs of computer.