https://github.com/tomviner/scrape-tory-nominations
A scraper that records various listings of declared Conservative nominations for leadership candidates
https://github.com/tomviner/scrape-tory-nominations
git-scraping politics
Last synced: 8 months ago
JSON representation
A scraper that records various listings of declared Conservative nominations for leadership candidates
- Host: GitHub
- URL: https://github.com/tomviner/scrape-tory-nominations
- Owner: tomviner
- Created: 2022-10-20T23:08:13.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-11-14T23:31:13.000Z (over 3 years ago)
- Last Synced: 2024-04-15T01:20:00.069Z (about 2 years ago)
- Topics: git-scraping, politics
- Language: Python
- Homepage:
- Size: 2.06 MB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Tory nomination scraper
A [git scraper](https://simonwillison.net/2020/Oct/9/git-scraping/) that records various spreadsheets containing listings of declared Conservative nominations for leadership candidates, so changes are visible.
## Data sources:
- Guido Fawkes [Tory Leadership: Who's Backing Who October Election](https://docs.google.com/spreadsheets/d/1PRufWhh2YAoxPUJEXeVaEOe7rcT1IINOXijeLI6o9Cc/htmlview)
Latest:
| | Rishi Sunak | Boris Johnson | Penny Mordaunt |
|---------------------|---------------|-----------------|------------------|
| 2022-10-24 12:52:01 | 202 | | 30 |
- [Smarkets Conservative leadership election tracker (October 2022)](https://docs.google.com/spreadsheets/d/1t1MaeGTmOvmOOkUL8TDDJwqTTc-N1wmRxPeRe0k3yjM/htmlview)
Latest:
| | Rishi Sunak | Boris Johnson | Penny Mordaunt |
|---------------------|---------------|-----------------|------------------|
| 2022-10-24 11:50:47 | 185 | 47 | 29 |
## Workflow
- download spreadsheet as csv
- convert to sqlite db with [git-history](https://pypi.org/project/git-history/) (see this repos requirement.txt to include https://github.com/simonw/git-history/pull/59)
- query db using [sqlite-utils](https://pypi.org/project/sqlite-utils/) to export time series
- generate graph
- inject latest total to the README with [cog](https://pypi.org/project/cogapp/) content generator
Attempts exports as CSV every 5 minutes, although Github actions seems to run less often than this.
Inspired by [Simon Willison](https://twitter.com/simonw/)'s [Half Moon Bay Pumpkin Festival scraper](https://github.com/simonw/scrape-hmb-traffic).