https://github.com/nickcrews/apoc-data
Data from the Alaska Public Offices Commission
https://github.com/nickcrews/apoc-data
campaign-finance playwright-python webscraping
Last synced: 21 days ago
JSON representation
Data from the Alaska Public Offices Commission
- Host: GitHub
- URL: https://github.com/nickcrews/apoc-data
- Owner: NickCrews
- Created: 2024-07-16T01:27:49.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2024-10-24T11:16:17.000Z (7 months ago)
- Last Synced: 2024-10-25T10:20:09.351Z (7 months ago)
- Topics: campaign-finance, playwright-python, webscraping
- Language: Python
- Homepage: https://aws.state.ak.us/ApocReports/Campaign/
- Size: 25.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# APOC Data
Data from the [Alaska Public Offices Commission](https://aws.state.ak.us/ApocReports/Campaign/).
This scrapes the CSV files from the APOC website once a day and uploads them to
[this repo's releases](https://github.com/NickCrews/apoc-data/releases).
## Manual
Browse from [this repo's releases](https://github.com/NickCrews/apoc-data/releases).
## Python
`pip install apoc-data` and then
```python
from apoc_data.download import download
download(
release="latest",
filename="debt.csv",
destination="apoc_debt.csv",
)
```
## Shell
You can download these CSVs using the direct URLs from the releases page
using curl, pandas, ibis, whatever!
```bash
curl -L https://github.com/NickCrews/apoc-data/releases/download/20240716-025636/candidate_registration.csv > candidate_registration.csv
```
or we have a tiny python script that makes this a little nicer, eg get the latest
release, choose the download directory, etc. Read the script for more info.
```bash
curl -s https://raw.githubusercontent.com/NickCrews/apoc-data/main/src/apoc_data/download.py | python - --release latest
```
## Dev Notes
```shell
pdm install
playwright install chromium
```
scrape:
```shell
python -m apoc_data.scrape --directory downloads --no-headless
```