{"id":19407505,"url":"https://github.com/aaronspring/remote_climate_data","last_synced_at":"2025-04-24T09:31:37.835Z","repository":{"id":37956333,"uuid":"283526954","full_name":"aaronspring/remote_climate_data","owner":"aaronspring","description":"a collection of remote climate data accessed via intake cached to disk","archived":false,"fork":false,"pushed_at":"2025-01-06T17:30:31.000Z","size":2471,"stargazers_count":19,"open_issues_count":11,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T02:23:13.640Z","etag":null,"topics":["accessibility","climate-data","climate-science","data-catalog","netcdf","observations","opendap","remote","shapefiles","thredds-catalogs"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aaronspring.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-29T14:52:05.000Z","updated_at":"2024-04-28T13:05:39.000Z","dependencies_parsed_at":"2024-01-13T07:48:41.861Z","dependency_job_id":"fac17668-990e-4c06-b99d-a33f83888d65","html_url":"https://github.com/aaronspring/remote_climate_data","commit_stats":{"total_commits":109,"total_committers":4,"mean_commits":27.25,"dds":"0.33944954128440363","last_synced_commit":"2d30a7b66b4ea82d8e72a120342f27d082cb6233"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronspring%2Fremote_climate_data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronspring%2Fremote_climate_data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronspring%2Fremote_climate_data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronspring%2Fremote_climate_data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aaronspring","download_url":"https://codeload.github.com/aaronspring/remote_climate_data/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250600707,"owners_count":21457012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accessibility","climate-data","climate-science","data-catalog","netcdf","observations","opendap","remote","shapefiles","thredds-catalogs"],"created_at":"2024-11-10T11:47:25.390Z","updated_at":"2025-04-24T09:31:36.532Z","avatar_url":"https://github.com/aaronspring.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![testing](https://github.com/aaronspring/remote_climate_data/actions/workflows/testing.yml/badge.svg)](https://github.com/aaronspring/remote_climate_data/actions/workflows/testing.yml) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/aaronspring/remote_climate_data/master?urlpath=lab%2Ftree%2Fnotebooks%2Fdemo.ipynb) [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/aaronspring/remote_climate_data/master.svg)](https://results.pre-commit.ci/latest/github/aaronspring/remote_climate_data/master)\n\n# remote_climate_data\na collection of remote climate data accessed via `intake` cached to disk\n\n## Usage\n```python\nimport intake\ncat = intake.open_catalog('https://raw.githubusercontent.com/aaronspring/remote_climate_data/master/master.yaml')\ncat.atmosphere.HadCRUT5.to_dask()\n```\n\nTo explore the whole catalog, you can try:\n```python\ncat.walk()\n```\n\n## Goal\nMake data access for climate data easy:\n- cacheable data\n- documentation attached in metadata\n- shareable catalogs\n- quick vizualisations\n\n## Contribute and extend\n- PRs for new remote climate datasets or useful geoshapes are very welcome\n\n## Relies on\n- [`intake`](https://intake.readthedocs.io/en/latest/) for catalogs and `csv` and [`zarr`](https://github.com/zarr-developers/zarr-python)\n- [`fsspec`](https://filesystem-spec.readthedocs.io/en/latest/features.html#url-chaining) for caching\n- [`intake_xarray`](https://intake-xarray.readthedocs.io/en/latest/) for:\n  - `nc` using [`netcdf4`](https://github.com/Unidata/netcdf4-python) [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/atmosphere.yaml#L64)]\n  - `tif` using [`rioxarray`](https://github.com/corteva/rioxarray) [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/humans.yaml#L42)]\n  - `grib` using [`cfgrib`](https://github.com/ecmwf/cfgrib/) [[example missing]()]\n- [`intake_thredds`](https://github.com/intake/intake-thredds) for using [`intake_xarray`](https://intake-xarray.readthedocs.io/en/latest/) via [THREDDS](https://www.unidata.ucar.edu/software/tds/current/) [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/atmosphere.yaml#L322)]\n- [`intake_excel`](https://github.com/edjdavid/intake-excel) for Excel `xls` and `xlsx` [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/climate.yaml#L35)]\n- [`intake_geopandas`](https://github.com/intake/intake_geopandas) for shapefiles `shp` [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/shapefiles.yaml#L11)], GeoJSON `geo.json` [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/shapefiles.yaml#L57)], GeoParquet `parquet`, `PostGIS` databases, `Spatialite` databases\n- [`regionmask`](https://regionmask.readthedocs.io/) for aggregating over geoshapes\n- [`hvplot`](https://hvplot.holoviz.org/index.html) for plotting [[example](https://github.com/aaronspring/remote_climate_data/blob/1209c5ebf5877b09b4403ea60da6d97b374b7b5c/catalogs/atmosphere.yaml#L48)]\n\n## Similar projects\n- Pangeo's cloud data catalogs for multi GB and TB datasets: https://github.com/pangeo-data/pangeo-datastore\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronspring%2Fremote_climate_data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaronspring%2Fremote_climate_data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronspring%2Fremote_climate_data/lists"}