https://github.com/rodekruis/ibf-drought-model
https://github.com/rodekruis/ibf-drought-model
aa
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/rodekruis/ibf-drought-model
- Owner: rodekruis
- Created: 2021-08-27T14:27:34.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2025-03-24T15:37:44.000Z (over 1 year ago)
- Last Synced: 2025-03-24T16:33:10.103Z (over 1 year ago)
- Topics: aa
- Language: Python
- Homepage:
- Size: 89.8 KB
- Stars: 1
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# IBF-drought-model
Drought forecast pipeline for Zimbabwe RCS.
## Introduction
This repo contains the code to:
1. Get latest ENSO data
2. Get latest CHIRPS data
3. Get latest VCI data
4. Forecast drought per province based on the latest data listed above.
5. Calculate impacts of drought province
Data is on MS Azure Datalake `ibf`.
## Functionalities
The pipeline is developed in Docker format. It can run locally in your desktop (with Dock installed, see below). A logic app is set up (510-ibf-drought) and to start the container and perform automated execution (monthly).
**`settings.py`** contains basic settings for the pipeline:
- To set test API for posting output and disable email notification
- To switch between operation and dummy mode using a boolean switch `dummy`
- To define lead-time based on the running mode and the month of execution
- To define a switch of the forecast model depending on the month of execution
- Data sources of ENSO, CHIRPS, VCI
**`utils.py`** contains main functions for the pipeline. For now in dummy mode, only the last 2 functions will be executed.
- `get_new_senso()`: get latest ENSO data from data source
- `get_new_chirps()`: get latest daily CHIRPS data from data source and calculate monthly accumulation and dryspell per district
- `get_new_vci()`: get latest observed VCI from data source and calculate monthly average VCI values per district
- `arrange_data()`: prepare an input data for model 2 by combining ENSO and CHIRPS (and VCI) data into one
- `forecast_model1()`: forecast drought per province based on the latest ENSO data, using trained XGBoost models
- `forecast_model2()`: forecast drought per province based on the latest ENSO, monthly rainfall and 14-day dry spell data, using trained XGBoost models
- `forecast_model3()`: forecast drought per province based on the latest ENSO, monthly rainfall, 14-day dry spell, and VCI data, using trained XGBoost models
- `calculate_impact()`: calculate exposed population, cattles, ruminants per drought-predicted province(s)
- `post_output()`: the processed data (drought forecast and impacts) will be posted to the IBF dashboard via IBF API
## Setup
### with Docker
1. Install [Docker](https://www.docker.com/get-started)
2. Build the docker image from the root directory
```
docker build -t rodekruis/ibf-drought-model .
```
3. Run and access the docker container
```
docker run -it --entrypoint /bin/bash rodekruis/ibf-drought-model
```
4. Set necessary credentials (in bitwarden) for the pipeline
## Usage
Run the pipeline with the command:
```
run-drought-model
```
## Versions
You can find the versions in the [tags](https://github.com/rodekruis/ibf-drought-model/tags) of the commits. See below table to find which version of the pipeline corresponds to which version of IBF-Portal.
| Drought Pipeline version | IBF-Portal version | Changes |
| --- | --- | --- |
| 0.2.6 | 0.305.2 | Convert to new warning/trigger setup |
| 0.2.4 | 0.261.5 | Update `python` env
Update `xgboost` package
Reorder input data in `arrange_data()`
Update objects of forecast model 2 and 3 based on new data order |
| 0.2.3 | 0.247.3 | Correct `leadtime_str` of API payload |
| 0.2.2 | 0.224.5 | Correct `arrange_data()` to get all past data
Add `date` to API payloads |
| 0.2.1 | 0.189.1 | Change VCI data name |
| 0.2.0 | 0.170.0 | Model 3 added to the pipeline
Function to download VCI data from NOAA added
Function to arrange data adjusted for VCI data
Enable downloading data in off-season|
| 0.1.3 | 0.152.0 | Add function to post non-trigger in off-season |
| 0.1.2 | 0.129.0 | Corrected generation of link to raw chirps file
Fixed misdownloading a processed rainfall from datalake
Fixed raw chirps files listing for calculating zonal statistics
Minor fixes |
| 0.1.1 | - | ENSO+rainfall model added
Minor fixes |
| 0.1.0 | - | Initial version, ENSO-only model |