https://github.com/machinecyc/lotteryinsight

Use crawler to collect Taiwan Lotto data, and save data into local MySQL server.
https://github.com/machinecyc/lotteryinsight

crawler data docker lottery mysql-database python3 taiwan

Last synced: 22 days ago
JSON representation

Use crawler to collect Taiwan Lotto data, and save data into local MySQL server.

• Host: GitHub
• URL: https://github.com/machinecyc/lotteryinsight
• Owner: machineCYC
• Created: 2021-04-05T03:00:21.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2022-08-27T06:21:47.000Z (about 2 years ago)
• Last Synced: 2023-03-07T11:52:18.304Z (over 1 year ago)
• Topics: crawler, data, docker, lottery, mysql-database, python3, taiwan
• Language: Python
• Homepage:
• Size: 126 KB
• Stars: 0
• Watchers: 2
• Forks: 1
• Open Issues: 3
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# LotteryInsight

This repository provide [Taiwan Lottery](https://www.taiwanlottery.com.tw/index_new.aspx) crawler framework to collect the data into mysql database.

Below table is the list of provide crawler.

Dataset       | Crawler | Official web update schedule | Crawler update scheduler |

--------------|:-----:|:-----:|:-----:|

今彩539    | DailyCash | Monday-Saturday 10pm | Monday-Saturday 10pm-11pm 1/20min |

大樂透   | Lotto649 | Tuesday, Friday 10pm | Tuesday, Friday 10pm-11pm 1/15min |

威力彩   | Superlotto638 | Monday, Thursday 10pm | Monday, Thursday 10pm-11pm 1/20min |

## Environment setting

    1. setup python env (must install pipenv pyenv)

        - pipenv --python /home/**username**/.pyenv/versions/3.6.9/bin/python3.6 shell

    2. sync python library

        - pipenv sync

    3. create .env file

        - python genenv.py

    4. setting environment variable in .env file

## Local develop setup

    1. create local mysql server

        - make run-dev-mysql

    2. develop the crawler in LotteryInsight/crawlers folder

    If you want to try regular crawler data, you can try step 3

    3. create local scheduler

        - make build-dev-image

        - make run-dev-scheduler

    4. If complete develop, please end up mysql server and scheduler.

        close mysql server

        - make end-dev-mysql

        close scheduler

        - make end-dev-scheduler