https://github.com/openclimatefix/gfs-downloader
NCEP GFS 0.25 Degree Global Forecast Grids Historical Archive: https://rda.ucar.edu/datasets/ds084.1/
https://github.com/openclimatefix/gfs-downloader
Last synced: 8 months ago
JSON representation
NCEP GFS 0.25 Degree Global Forecast Grids Historical Archive: https://rda.ucar.edu/datasets/ds084.1/
- Host: GitHub
- URL: https://github.com/openclimatefix/gfs-downloader
- Owner: openclimatefix
- License: mit
- Created: 2023-04-06T16:29:59.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-06-22T16:13:54.000Z (almost 3 years ago)
- Last Synced: 2025-01-20T23:58:16.025Z (over 1 year ago)
- Language: Python
- Size: 146 KB
- Stars: 4
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# gfs-downloader
NCEP GFS 0.25 Degree Global Forecast Grids Historical Archive: https://rda.ucar.edu/datasets/ds084.1/
Register and make an account here: https://rda.ucar.edu/login/
Create a .env file with this in (make sure to name the file .env and leave it in the root directory):
To create a .env file do:
```
nano .env
```
Then add your credentials to the file
```
UCAR_EMAIL = "INSERT_EMAIL"
UCAR_PASS = "INESERT_PASSWORD"
```
### Setting up the environment
```
pip install -r requirements.txt
```
### Instal the ecCodes package
```
conda install -c conda-forge python-eccodes
```
Add others (dask, etc...)
### Download Operation
To see download script inputs: `poetry run Island_nwp_ext_latlon.py --help`
**Example [DEMO] (if using poetry):**
`poetry run python Island_nwp_ext_latlon.py Start_Date End_Date Output_Path LATMIN LATMAX LONGMIN LONGMAX`
or (for normal venv)
```
python Island_nwp_ext_latlon.py Start_Date End_Date Output_Path LATMIN LATMAX LONGMIN LONGMAX
```
Date format YYYY-MM-DD
To keep operation running in the background use nohup (a nohup.out file will be created to keep a log):
```
nohup python Island_nwp_ext_latlon.py Start_Date End_Date Output_Path LATMIN LATMAX LONGMIN LONGMAX &
```
Set the location of the output log, this example will create the nwp_run_v1.log at the location where the command was executed:
```
nohup python Island_nwp_ext_latlon.py Start_Date End_Date Output_Path LATMIN LATMAX LONGMIN LONGMAX >> nwp_run_v1.log 2>&1 &
```
Watch the log in the CLI live:
```
tail -f nwp_run_v1.log
```
Look at all the full logs with:
```
nano nwp_run_v1.log
```
Monitor scripts executing and uptime (This is also where you can find the PID in case the need to Kill a script) [Kill PID]:
```
ps aux | grep Island_nwp_ext_latlon.py
```
### Multitasks using parallel:
First instal parallel:
```
conda install parallel
```
##### Generating dates:
Use the script `date_generator.py` to generated a "date_chunks" variable.
```
python date_generator.py Start_Date End_Date Num_Parallel
```
or manually set running:
```
date_chunks=$(cat <> nwp_run_p1.log 2>&1 &
```