https://github.com/rvegas/dota_crawler

Crawler for dotapedia. Fills a Mongo and a PG database with game data.
https://github.com/rvegas/dota_crawler

crawler dota dota2 flask mongodb postgresql python3 regex scrapy

Last synced: 5 days ago
JSON representation

Crawler for dotapedia. Fills a Mongo and a PG database with game data.

• Host: GitHub
• URL: https://github.com/rvegas/dota_crawler
• Owner: rvegas
• Created: 2017-07-10T18:16:50.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2017-10-16T11:07:05.000Z (about 7 years ago)
• Last Synced: 2023-08-15T15:27:31.443Z (over 1 year ago)
• Topics: crawler, dota, dota2, flask, mongodb, postgresql, python3, regex, scrapy
• Language: Python
• Homepage:
• Size: 2.8 MB
• Stars: 5
• Watchers: 1
• Forks: 5
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Dota Crawler

Please run:

```

sudo pip install virtualenv

virtualenv .crawler

source .crawler/bin/activate

pip install -r requirements.txt

scrapy runspider herocrawl.py

scrapy runspider itemcrawl.py

```