https://github.com/nathanthorell/snowfall-analytics
NOAA snowfall data analytics pipeline
https://github.com/nathanthorell/snowfall-analytics
dbt duckdb evidence-dev noaa-data python
Last synced: over 1 year ago
JSON representation
NOAA snowfall data analytics pipeline
- Host: GitHub
- URL: https://github.com/nathanthorell/snowfall-analytics
- Owner: nathanthorell
- License: gpl-3.0
- Created: 2025-02-10T18:47:19.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-11T01:16:38.000Z (over 1 year ago)
- Last Synced: 2025-03-11T02:24:52.448Z (over 1 year ago)
- Topics: dbt, duckdb, evidence-dev, noaa-data, python
- Language: Python
- Homepage:
- Size: 31.3 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# snowfall-analytics
This project extracts snowfall data from NOAA's Climate Data API, processes it using DuckDB and dbt, and presents analytics through [Panel](https://panel.holoviz.org/) dashboards.
More documentation coming as the project develops.
## Setup
### Prerequisites
- Python 3.12+
- Make
### Initial Setup
1. Clone the repository:
```bash
git clone https://github.com/nathanthorell/snowfall-analytics.git
cd snowfall-analytics
```
1. Create and install the development environment:
```bash
make install
source .venv/bin/activate
```
## Common Commands
```bash
make lint # Run code quality checks
make format # Format code
make test # Run tests
make all # Run format, lint, and test
```
## Project Structure
```text
snowfall-analytics/
├── src/
│ └── snowfall_analytics/
│ ├── cli.py # Command-line interface
│ ├── config.py # Configuration management
│ ├── noaa.py # NOAA API client and models
│ ├── sql_loader.py # loads dir of sql files
│ └── db.py # DuckDB operations
├── tests/ # Test files
├── dbt_snowfall/ # dbt transformations
├── dashboards/ # Panel dashboards
├── pyproject.toml # Python package configuration
└── Makefile # Build and development commands
```