https://github.com/stalker314314/wiki-climate

Climate data scraper from Wikipedia/Wikidata
https://github.com/stalker314314/wiki-climate

climate climate-data pymongo python python-3 pywikibot requests sparql wiki wikidata wikipedia wikipedia-api

Last synced: about 1 month ago
JSON representation

Climate data scraper from Wikipedia/Wikidata

• Host: GitHub
• URL: https://github.com/stalker314314/wiki-climate
• Owner: stalker314314
• License: gpl-3.0
• Created: 2018-09-21T18:26:55.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-10-12T20:12:07.000Z (about 6 years ago)
• Last Synced: 2024-11-04T08:35:22.540Z (2 months ago)
• Topics: climate, climate-data, pymongo, python, python-3, pywikibot, requests, sparql, wiki, wikidata, wikipedia, wikipedia-api
• Language: Python
• Size: 20.5 KB
• Stars: 4
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-starred - stalker314314/wiki-climate - Climate data scraper from Wikipedia/Wikidata (python)

README

        
# Wiki climate

Tool with which climate data can be extracted from Wikipedia for all cities.

Accompanying code to analysis [what is best city to live in](https://www.kaggle.com/brankokokanovic/best-city-to-live-in). Latest (October 2018) data can be seen at [Kaggle](https://www.kaggle.com/brankokokanovic/wiki-climate/home).

Works by obtaining cities using SPARQL/Wikidata and then query Wikipedia (using pywikibot) to parse those weatherbox boxes.

Data is stored in Mongo and can be exported to JSON with:

```

mongo.exe --quiet temp --eval "printjson(db.cities.find({}, {'_id':0}).toArray())" > wiki-climate.json

```