{"id":37634516,"url":"https://github.com/rmcd-mscb/gridmet-etl","last_synced_at":"2026-01-16T11:01:45.144Z","repository":{"id":230939844,"uuid":"780503724","full_name":"rmcd-mscb/gridmet-etl","owner":"rmcd-mscb","description":null,"archived":false,"fork":false,"pushed_at":"2024-11-08T17:15:09.000Z","size":48266,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-08T18:24:34.172Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rmcd-mscb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-01T16:09:53.000Z","updated_at":"2024-11-08T17:15:12.000Z","dependencies_parsed_at":"2024-04-12T23:24:59.087Z","dependency_job_id":"8e8bddf9-e2ba-435b-83e7-93122e6f27ec","html_url":"https://github.com/rmcd-mscb/gridmet-etl","commit_stats":null,"previous_names":["rmcd-mscb/gridmet-etl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rmcd-mscb/gridmet-etl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmcd-mscb%2Fgridmet-etl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmcd-mscb%2Fgridmet-etl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmcd-mscb%2Fgridmet-etl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmcd-mscb%2Fgridmet-etl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rmcd-mscb","download_url":"https://codeload.github.com/rmcd-mscb/gridmet-etl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmcd-mscb%2Fgridmet-etl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T06:30:42.265Z","status":"ssl_error","status_checked_at":"2026-01-16T06:30:16.248Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-16T11:01:45.051Z","updated_at":"2026-01-16T11:01:45.126Z","avatar_url":"https://github.com/rmcd-mscb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gridmet-etl\n\nA python package and CLI for processing area-weighted spatial interpolations of gridmet climate drivers using gdptools.  This package is intended to be used for processing gridmet climate drivers for the U.S. Geolocical Survey's Operational National Hydrologic Model.  \n\nCurrently the packages supports processing [gridmet](https://www.climatologylab.org/gridmet.html) and downscaled [CFSv2](http://thredds.northwestknowledge.net:8080/thredds/catalog/NWCSC_INTEGRATED_SCENARIOS_ALL_CLIMATE/cfsv2_metdata_90day/catalog.html).  For the latter it processes the 48 day ensemble versions and can process either the ensemble median, or all 48 ensembles.  See below for examples.\n\n## Install\n\nTo install the package into a local conda env, run the following commands.\n\n```console\nconda env create -f environment.yml\nconda activate gridmet-etl\npoetry install\n```\n\n## Example code usage\n\nTo see available commands, run `gridmetetl -h`\n\n```console\n\u0026 gridmetetl -h\nUsage: gridmetetl COMMAND\n\n╭─ Commands ──────────────────────────────────────────────────────────────╮\n│ cfsv2-etl                                                               │\n│ gridmet-etl                                                             │\n│ --help,-h    Display this message and exit.                             │\n│ --version    Display application version.                               │\n╰─────────────────────────────────────────────────────────────────────────╯\n```\n\n### gridmet\n\nTo see the interface for running a gridmet-etl command, run `gridmetetl gridmet-etl -h`\n\n```console\nsage: gridmetetl gridmet-etl [ARGS] [OPTIONS]\n\nRetrieves and processes environmental data within a specified date range\nand geographical area, then outputs the results as netCDF files.\n\n╭─ Parameters ────────────────────────────────────────────────────────────╮\n│ *  START-DATE,--start-date                        Start date of         │\n│                                                   retrieval             │\n│                                                   (YYYY-MM-DD)          │\n│                                                   [required]            │\n│ *  END-DATE,--end-date                            Start date of         │\n│                                                   retrieval             │\n│                                                   (YYYY-MM-DD)          │\n│                                                   [required]            │\n│ *  FILE-PREFIX,--file-prefix                      Prefix for output     │\n│                                                   files [required]      │\n│ *  TARGET-FILE,--target-file                      Input geometry file   │\n│                                                   (target polygon       │\n│                                                   geometry file: read   │\n│                                                   by geopandas)         │\n│                                                   [required]            │\n│ *  OUTPUT-PATH,--output-path                      Output path (location │\n│                                                   of netcdf output      │\n│                                                   files by shapefile    │\n│                                                   output) [required]    │\n│ *  WEIGHT-FILE,--weight-file                      path/weight.csv -     │\n│                                                   path/name of weight   │\n│                                                   file [required]       │\n│ *  FID,--fid                                      Target file column id │\n│                                                   to identify results   │\n│                                                   [required]            │\n│    PARTIAL,--partial,--no-partial                 option: set if you    │\n│                                                   expect only partial   │\n│                                                   mapping to HRU and    │\n│                                                   you want the partial  │\n│                                                   mapping [default:     │\n│                                                   False]                │\n│    FILL-MISSING,--fill-missing,--no-fill-missing  option: fill outside  │\n│                                                   conus hrus according  │\n│                                                   to nearest neighbor   │\n│                                                   using mapping file    │\n│                                                   [default: False]      │\n╰─────────────────────────────────────────────────────────────────────────╯\n```\n\n* example command:\n\n  ```console\n  gridmetetl gridmet-etl --start-date 1980-01-01 --end-date 1980-01-07 --file-prefix gm_ --target-file ./data/nhru_01a/nhru_01a.parquet --output-path ./data/output --weight-file ./data/nhru_01a/gm_nhru01_weights.csv --fid nhru_v1_1 --fill-missing --no-partial\n  ```\n\n* result: This results in a file named data/output/1980-01-01_converted_filled.nc\n* [result header](gm_header.md)\n\n### cfsv2\n\nTo see the interface for running a cfsv2-etl command, run `gridmetetl cfsv2-etl -h`\n\n```console\nUsage: gridmetetl cfsv2-etl [ARGS] [OPTIONS]\n\n╭─ Parameters ────────────────────────────────────────────────────────────────────╮\n│ *  FILE-PREFIX,--file-prefix                      Prefix for output files       │\n│                                                   [required]                    │\n│ *  TARGET-FILE,--target-file                      Input geometry file (target   │\n│                                                   polygon geometry file: read   │\n│                                                   by geopandas) [required]      │\n│ *  FID,--fid                                      Target file column id to      │\n│                                                   identify results [required]   │\n│ *  OUTPUT-PATH,--output-path                      Output path (location of      │\n│                                                   netcdf output files by        │\n│                                                   shapefile output) [required]  │\n│ *  WEIGHT-FILE,--weight-file                      path/weight.csv - path/name   │\n│                                                   of weight file [required]     │\n│ *  METHOD,--method                                Choice of (1) median or (2)   │\n│                                                   ensemble [required]           │\n│    PARTIAL,--partial,--no-partial                 option: set if you expect     │\n│                                                   only partial mapping to HRU   │\n│                                                   and you want the partial      │\n│                                                   mapping [default: False]      │\n│    FILL-MISSING,--fill-missing,--no-fill-missing  option: fill outside conus    │\n│                                                   hrus according to nearest     │\n│                                                   neighbor using mapping file   │\n│                                                   [default: False]              │\n╰─────────────────────────────────────────────────────────────────────────────────╯\n```\n\n#### Method: Ensemble_median\n\nThis produces an interpolation of the median of all 48 ensembles for tmax, tmin, and precip. The file prefix is the start_date of the forecast.  For example `2024-04-01_converted_filled.nc`\n\n* example command using `--method 1` for the ensemble median:\n\n  ```console\n  gridmetetl cfsv2-etl --file-prefix cfsv2_ --target-file ./data/nhru_01a/nhru_01a.parquet --output-path ./data/output --weight-file ./data/nhru_01a/gm_nhru01_weights.csv --fid nhru_v1_1 --method 1 --fill-missing --no-partial\n  ```\n\n* result: This results in a file named data/output/median_ensemble/2024-04-01_converted_filled.nc\n* [result header](cfsv2_median_header.md)\n\n#### Method: Ensemble\n\nThis produces an interpolation for each of the 48 ensembles for tmax, tmin, precip.  The file prefix is the start_date of the forecast and the file post-fix is the ensemble number, for esample: `2024=04-01_converted_filled_0.nc`\n\n* example command using `--method 2` for the ensemble median:\n\n  ```console\n  gridmetetl cfsv2-etl --file-prefix cfsv2_ --target-file ./data/nhru_01a/nhru_01a.parquet --output-path ./data/output --weight-file ./data/nhru_01a/gm_nhru01_weights.csv --fid nhru_v1_1 --method 2 --fill-missing --no-partial\n  ```\n\n* result: This results in a file named data/output/ensembles/2024-04-01_converted_filled_0.nc\n* [result header](cfsv2_ensemble_header.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmcd-mscb%2Fgridmet-etl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frmcd-mscb%2Fgridmet-etl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmcd-mscb%2Fgridmet-etl/lists"}