https://github.com/froukje/cygnss-deployment
Web Interface for Wind Speed Prediction with a Neural Net
https://github.com/froukje/cygnss-deployment
cygnss deep-learning docker-compose mongodb prefect streamlit windspeed
Last synced: 3 months ago
JSON representation
Web Interface for Wind Speed Prediction with a Neural Net
- Host: GitHub
- URL: https://github.com/froukje/cygnss-deployment
- Owner: froukje
- Created: 2022-12-12T12:14:25.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-12-13T16:01:45.000Z (over 2 years ago)
- Last Synced: 2025-01-13T20:29:09.179Z (5 months ago)
- Topics: cygnss, deep-learning, docker-compose, mongodb, prefect, streamlit, windspeed
- Language: Jupyter Notebook
- Homepage:
- Size: 1.62 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Web Interface for Wind Speed Prediction
### About
The objective of this repository is to deploy a pre-trained *CyGNSSnet* to predict global ocean wind speed in near time. The results are shown on a web interface, which provides different illustrations of the predicted wind speed and its error compared to [ERA5 windspeed](https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5) data.
*CyGNSSnet* is a neural net developed to predict wind speed from [CYGNSS](https://podaac.jpl.nasa.gov/dataset/CYGNSS_L2_V3.0)(**Cy**clone **G**lobal **N**avigation **S**atellite **S**ystem) data. The code for *CyGNSSnet* itself is not public. For more information or if you need to access it contact Caroline Arnold ([email protected]) or the Helmholtz AI consultant team for Earth and Environment ([email protected]). For more information on *CyGNSSnet*, see [Asgarimehr et al, Remote Sensing of Environment (2022)](https://doi.org/10.1016/j.rse.2021.112801)
### Workflow
### Quick start
To start the deployment run ```sh set_up_infrastructure.sh```.
This clones the git repository and starts the deployment using docker-compose.
Make sure you have docker and docker-compose installed.
If you have already the cloned the git repository move to the directory ```docker_cygnss_deployment``` and run
```
docker-compose up
```
To stop the container, run following command:
```
docker-compose -f ./docker-compose.yml down --remove-orphans
```
Note: In order to run it you need access to the external submodule containing the CyGNSSnet.
The deployment is scheduled using prefect. It is executed every day and downloads the CyGNSS data for the current date minus 10 days. Then the predictions are calculated, stored in a mongodb database and displayed on a streamlit dashboard.
To access the streamlit dashboard: http://localhost:8501
To access the mongodb database: http://localhost:8081
To access the prefect ui: http://localhost:5000
### Repository Structure
```
API.py: download CyGNSS data
Preprocessing.py: download ERA5 data and preprocess data
dashboard.py: streamlit dashboard
plots.py: helper functions to create the plots for the streamlit dashboard
prefect-deploy.py: Deployment scheduled for every day
externals/: folder with CyGNSSnet code
notebooks/: folder with some notebooks that were created during the development
docker_cygnss_deployment/: folder with docker files to start deployment
```
## Data source
- CYGNSS. CYGNSS Level 2 Science Data Record Version 3.1. Ver. 3.1. PO.DAAC, CA, USA. accessed 2022/2023 at 10.5067/CYGNS-L2X31
- Copernicus Climate Change Service (C3S) (2017): ERA5: Fifth generation of ECMWF atmospheric reanalyses of the global climate . Copernicus Climate Change Service Climate Data Store (CDS), 2022/2023. https://cds.climate.copernicus.eu/cdsapp#!/home