{"id":23278505,"url":"https://github.com/hdfgroup/ghcn_collector","last_synced_at":"2025-04-06T12:42:07.865Z","repository":{"id":66041896,"uuid":"425969506","full_name":"HDFGroup/ghcn_collector","owner":"HDFGroup","description":"Convert GHCN weather data CSV files to HDF","archived":false,"fork":false,"pushed_at":"2022-03-30T21:02:36.000Z","size":40,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-12T18:32:10.844Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HDFGroup.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-08T19:36:08.000Z","updated_at":"2021-11-16T16:09:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"47efb9ce-9d13-462d-96b6-17c0c88e1765","html_url":"https://github.com/HDFGroup/ghcn_collector","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fghcn_collector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fghcn_collector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fghcn_collector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HDFGroup%2Fghcn_collector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HDFGroup","download_url":"https://codeload.github.com/HDFGroup/ghcn_collector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247485279,"owners_count":20946397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-19T22:32:02.856Z","updated_at":"2025-04-06T12:42:07.832Z","avatar_url":"https://github.com/HDFGroup.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"GHCN Data Loader\n======================================================================\n \nIntroduction\n------------\n\nThis project can be used to create an HDF5 file containing GHCN (Global Historical Climatology Network Daily) data.  NOAA updates GHCN records daily for thousands\nof weather stations across the globe.  These records are available on AWS S3 as CSV\nfiles stored by year (starting in 1876!).  The data loader will incremental pull data\nfrom these files and construct an HDF5 dataset containing all the records.  Once created,\nyou can use the HDF5 library or HSDS to query and analyze information from the file.\n\nUsage\n-----\n\nUpdate config.yml with values for filename, and any other desired values. To use HSDS \nrather than HDF5 library, add the \"hdf5://\" prefix to the filepath.\nHSDS endpoint, username, and password can be specified here, or pulled from a .hscfg file,\nor specified using environment variables (`export HSDS_ENDPOINT=http://myhsds.myorg.org`, etc.)\n\nRun: `python ghcn_setup.py \u003cfilepath\u003e` to initialize the HDF5 or HSDS domain file.\n\nRun: `python ghcn_update.py` to start the collection of GHCN data.  If the `run_forever`\nconfig is set, the script will periodically check for updates in the GHCN CSV bucket.  Otherwise, it will stop after the desired year range is collected.\n\nRelated Information\n--------------------\n\nSee: \u003chttps://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn\u003e for \ninformation on the data format.\n\nThe AWS Open Data repository for GHCN data is described here: \u003chttps://registry.opendata.aws/noaa-ghcn/\u003e.\n\nInformation about HSDS can be found here: \u003chttps://github.com/HDFGroup/hsds\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhdfgroup%2Fghcn_collector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhdfgroup%2Fghcn_collector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhdfgroup%2Fghcn_collector/lists"}