https://github.com/bitbravo/python-scrape
https://github.com/bitbravo/python-scrape
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/bitbravo/python-scrape
- Owner: BitBravo
- Created: 2020-03-05T01:38:30.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-04-27T08:24:48.000Z (about 6 years ago)
- Last Synced: 2025-01-02T04:28:01.034Z (over 1 year ago)
- Language: Python
- Size: 9.55 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# Business Deal Scrapper
### Installation
**- Set up Pipenv on your machine**
```
pip install pipenv
```
**- Enable virtualenv using Pipenv**
```
pipenv shell
```
**- Install python libraries**
```
pipenv install
```
**- AWS S3 configuration**
Set AWS configuration in config.ini
```
[AWS]
ACCESS_KEY: your aws access key
SECRET_KEY: your aws secrted key
REGION: your aws regison
BUCKET: S3 Bucket name
```
### How to run.
```
$ cd project_root
$ python bot.py
```
### Properties
It will run this script twice in a week.
To change this schedule times, please config it in bot.py.
-> Tuesday 18:00
-> Friday 18:00
> ***If you want to test it quickly, please disable the Time Schedule Block and enable Quick Test blok in bot.py file.***
This is a Time Schedule Block.
````
# Time Schdule
schedule.every().tuesday.at("18:00").do(main)
schedule.every().friday.at("18:00").do(main)
while True:
schedule.run_pending()
sleep(10)
````
Quick Test Block.
````
# Quick TEST
# main()
````