https://github.com/nickhand/doge-savings-scraper
Scraping CFPB contracts from doge.gov/saving
https://github.com/nickhand/doge-savings-scraper
Last synced: 8 months ago
JSON representation
Scraping CFPB contracts from doge.gov/saving
- Host: GitHub
- URL: https://github.com/nickhand/doge-savings-scraper
- Owner: nickhand
- Created: 2025-03-07T16:41:10.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-15T01:59:58.000Z (over 1 year ago)
- Last Synced: 2025-03-15T02:36:18.658Z (over 1 year ago)
- Language: Python
- Size: 192 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Scraping doge.gov/savings
## Install
```
git clone https://github.com/nickhand/doge-savings-scraper.git
cd doge-savings-scraper
poetry install
```
## Run
```
poetry run doge-savings-scraper run --browser firefox --log-freq 10
```
This saves the scraping results to a CSV file in the `data` directory. The `--log-freq` flag specifies how often to log the scraping progress. You can choose "firefox" or "chrome" for the `--browser` flag.
## Daily scrape
A GitHub action workflow runs once a day to scrape the data and commit the results to this repository. The workflow is defined in `.github/workflows/scrape.yml`. The script saves the results to the `data` directory, tagged with the timestamp of the scrape.