Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tebben/kanemi
Tools for working with KNMI and some other data
https://github.com/tebben/kanemi
cli grib hdf5 knmi netcdf nowcast pdok rust svelte tauri
Last synced: 22 days ago
JSON representation
Tools for working with KNMI and some other data
- Host: GitHub
- URL: https://github.com/tebben/kanemi
- Owner: tebben
- License: mit
- Created: 2024-12-04T18:58:55.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2024-12-31T17:57:25.000Z (23 days ago)
- Last Synced: 2024-12-31T18:29:32.185Z (23 days ago)
- Topics: cli, grib, hdf5, knmi, netcdf, nowcast, pdok, rust, svelte, tauri
- Language: Rust
- Homepage:
- Size: 10.6 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
The goal of kanemi is to provide libraries and tools to work more easily with KNMI data.
> Warning: LOOK MA! I'M DOING RUST!!!! (This repo is a learning project for Rust, no guarantees for the quality)
## Testimonials
- "I'm not sure what this is, but it looks cool" - Mom
- "Since I started using kanemi, my wife and dog moved back in with me" - TheDude22
- "kanemi told me it was going to rain. I went out without an umbrella. It didn’t rain. So... thanks for making me look like an idiot." - StevenSeagul
- "kanemi: Finally, a way to make forecasts as accurate as flipping a coin." - NotImpressed44
- "LET'S GO!!! PRICIPITATION FORECASTING AND MUCH MORE WITH KANEMI, BABY!!!" - SteveBall
## Datasets
Kanemi tries to provide easy access to the following datasets
### Nowcast precipitation forecast
[Nowcast precipitation forecast](https://dataplatform.knmi.nl/dataset/radar-forecast-2-0) up to 2 hours ahead, per 5 minutes, over the Netherlands. The forecast contains 25 time steps: +0 minutes to +120 minutes. Forecasted is the precipitation sum per 5 minutes, on a grid of 1x1 km. The forecast is made with an operational KNMI implementation of pySTEPS. The forecast is initiated with the KNMI 5-minute real-time precipitation accumulation product: RTCOR-5m.
### HARMONIE-AROME Cy43 forecasts Netherlands (P1)
HARMONIE information in gridded form (regular lat-lon) of near surface and boundary layer (up to 300 m) parameters from the UWC-West HARMONIE-AROME Cy43 model. For this weather forecast model, KNMI works closely with Iceland, Denmark and Ireland on local short-term weather forecasts under the name "United Weather Centres-West" (UWC-West). An international project team is working on a joint Numerical Weather Prediction model (NWP), procurement and management of the HPC (supercomputer) and infrastructure. The output frequency is 1 hour.
### Meteo data - actual synoptic observations
KNMI collects observations from the automatic weather stations situated in the Netherlands and BES islands on locations such as aerodromes and North Sea platforms. In addition, wind data from KNMI wind poles are included. The weather stations report every 10 minutes meteorological parameters such as temperature, relative humidity, wind, air pressure, visibility, precipitation and cloud cover. The number of parameters differs per station. The file for the past 10-minutes is available a few minutes later. It is possible that a station's observations may not be immediately available. Files are updated with missing data up to 4 hours later.
## Features
This repo consists of multiple libraries and tools. Everythng is work in progress.
### Library kanemi
- Working with KNMI Data Platform
- Receiving notifications from the KNMI notification service
- Nowcast precipitation forecast
- Load dataset from HDF5
- Projection conversion between HDF5 grid and EPSG:4326 (both directions)
- Read image data and their time attribute
- Get pixel values and mm/hr for a specific location (xy/lonlat)
- Get 2hr precipitation forecast for a specific location (lonlat)
- Actual synoptic observations
- Load dataset from netCDF
- Get all stations and their observations
- Get closest station with observations and distance for a given location (lonlat)
- Harmonie Cy43 P1
- Work in progress: Grib v1 reader optimized for Cy43 P1
### Library pdok_geocoder
- Geocode free text search
- Reverse geocode lonlat/rd
### CLI tool kanecli
- Download KNMI data from the KNMI Data Platform
- Get notifications from the KNMI notification service
- Get precipitation forecast for a specific location
- Geocode and reverse geocode locations in the Netherlands
### App
- Desktop app using Tauri and Svelte (work in progress)
## Roadmap
- Desktop app using Tauri and Svelte
- Make CLI better and add more features
- Geocoder: Lookup, suggets
- More datasets
- Webservice/API
## Prerequisites
### HDF5
HDF5 library is required to run and build knmi-nowcast. Install it with:
```bash
sudo apt install libhdf5-dev
```
### KNMI API key
#### Open Data API
To fetch the nowcast data, you need an API key from the KNMI. You can find more information [here](https://developer.dataplatform.knmi.nl/open-data-api#token). An anonymous key is available on the mentioned page aswell.
#### Notifications
For the notifications service a different API key is needed and can be requested like the Open Data API key.
## kanecli - CLI Tool
For now the CLI only contains 4 commands to download datasets, get notifications, get precipitation forecast and (reverse)geocoding. Every command accepts it's own set of args, check the args with the `--help` flag on an option. `kanecli forecast --help` for example.
```bash
A CLI tool to work with KNMI data and maybe some other stuff
Usage: kanecli
Commands:
download Download KNMI data from the KNMI Data Platform
forecast Print the precipitation forecast from input file or a newly downloaded KNMI dataset
notifications Test the notification service
geocoder Geocode or reverse geocode a location using the PDOK Locatieserver
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
```
### Installation
To install kanecli, clone the repository and build the project with cargo. This will install the binary in the cargo bin directory (usually `~/.cargo/bin`).
```bash
git clone https://github.com/tebben/kanemi.git
cd kanemi
cargo install --path kanecli
```
### Examples
Download the latest forecast and return the precipitation forecast for the given location.
```bash
kanecli forecast -a -l ","
```
Get notification messages for the dataset radar_forecast version 2.0.
```bash
kanecli notifications -a --dataset-name radar_forecast --dataset-version 2.0
```
Geocode a location using the PDOK Locatieserver, filter on address type and return only the best match.
```bash
kanecli geocoder free -q "museumstraat 1 Amsterdam" --fq "type:adres" --best-match
```
Reverse geocode, find the 5 best adres matches for the given lonlat within 15 meters.
```bash
kanecli geocoder reverse --lonlat "4.887295127944717,52.36849110206849" --rows 5 --distance 15
```
### Building the CLI
```bash
cargo build --release
```
## App
Testing creating an app using Tauri with Svelte
```sh
cd app
cargo tauri dev
```