https://github.com/ngshiheng/cafireshistorydb
Tracking fire data from www.fire.ca.gov
https://github.com/ngshiheng/cafireshistorydb
datasette disaster fires ghactions-scraping git-scraping
Last synced: 6 months ago
JSON representation
Tracking fire data from www.fire.ca.gov
- Host: GitHub
- URL: https://github.com/ngshiheng/cafireshistorydb
- Owner: ngshiheng
- License: mit
- Created: 2024-07-01T08:31:49.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-04-19T04:22:20.000Z (7 months ago)
- Last Synced: 2025-05-11T01:11:13.996Z (6 months ago)
- Topics: datasette, disaster, fires, ghactions-scraping, git-scraping
- Language: Makefile
- Homepage: https://cafireshistory.jerrynsh.com
- Size: 65.4 KB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# California Fires History DB
Tracking fire data from www.fire.ca.gov/incidents.
This project is inspired by and adapts the work from [simonw/ca-fires-history](https://github.com/simonw/ca-fires-history). Instead of using [Git scraping](https://simonwillison.net/2020/Oct/9/git-scraping/), this project stores the data in a SQLite database and uses GitHub artifacts for storage.
[Read more...](https://jerrynsh.com/how-i-saved-scraped-data-in-an-sqlite-database-on-github/)
## How It Works
```mermaid
graph TB
subgraph Railway
deployment[Datasette]
end
subgraph GitHub
subgraph Actions
scraper[scrape.py]
end
subgraph Artifacts
db[(fires.db)]
class db artifacts;
end
end
subgraph CAL FIRE
api[API]
end
subgraph DockerHub
dockerhub[Docker Hub]
end
db --> |1: Download| scraper
api --> |2: Fetch Data| scraper
scraper --> |3: Upload| db
scraper --> |4: Publish to Docker Hub| dockerhub
dockerhub --> |5: Pull Image and Deploy| deployment
deployment --> |6: View/Access Data| client[User]
```
## Usage
### Running the Script
To run the script locally and update the database:
```bash
python3 scrape.py
```
### Optional: Running Datasette
To view and explore the data using [Datasette](https://datasette.io/):
```sh
pip install datasette # optional: install Datasette if you haven't already
datasette data/fires.db --metadata data/metadata.json
```
## License
This project is licensed under the [MIT License](./LICENSE).
## Disclaimer
This software is only used for research purposes, users must abide by the relevant laws and regulations of their location, please do not use it for illegal purposes. The user shall bear all the consequences caused by illegal use.