{"id":30201484,"url":"https://github.com/esipfed/global-lake-level-database","last_synced_at":"2025-08-22T12:43:44.063Z","repository":{"id":45526916,"uuid":"245296712","full_name":"ESIPFed/Global-Lake-Level-Database","owner":"ESIPFed","description":"A dev for ESIP grant","archived":false,"fork":false,"pushed_at":"2022-02-14T15:48:17.000Z","size":5664,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-08-13T10:51:28.743Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ESIPFed.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-06T00:31:35.000Z","updated_at":"2024-01-10T05:11:19.000Z","dependencies_parsed_at":"2022-07-18T23:02:34.828Z","dependency_job_id":null,"html_url":"https://github.com/ESIPFed/Global-Lake-Level-Database","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ESIPFed/Global-Lake-Level-Database","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESIPFed%2FGlobal-Lake-Level-Database","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESIPFed%2FGlobal-Lake-Level-Database/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESIPFed%2FGlobal-Lake-Level-Database/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESIPFed%2FGlobal-Lake-Level-Database/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ESIPFed","download_url":"https://codeload.github.com/ESIPFed/Global-Lake-Level-Database/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ESIPFed%2FGlobal-Lake-Level-Database/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271641792,"owners_count":24795436,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-22T02:00:08.480Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-13T10:50:15.546Z","updated_at":"2025-08-22T12:43:44.024Z","avatar_url":"https://github.com/ESIPFed.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Global Lake Level Database\n\n![](docs/ESIP_Lakes_Cover.jpg)\n\nThe **Global Lake Level Database (GLLD)** is the back-end architecture for the Python package [LakePy](https://github.com/ESIPFed/LakePy).\nThe GLLD hosts historic lake level data on an Amazon Web Services (AWS) Relational Database Service (RDS). Currently the\n GLLD data comes from three sources (so far!)\n\n - [United States Geological Survey National Water Information System](https://waterdata.usgs.gov/nwis)\n - [United States Department of Agriculture: Foriegn Agricultural Service's G-REALM Database](https://ipad.fas.usda.gov/cropexplorer/global_reservoir/)\n - [Theia's HydroWeb Database](http://hydroweb.theia-land.fr/)\n\n**Funding for this work comes from the Earth Science Information Partners (ESIP) Winter 2020 Grant**\n\n_See the funded proposal [here](https://www.esipfed.org/wp-content/uploads/2020/04/Gearon.pdf)_\n\n## The Basics\n\nThe GLLD consists of two tables:\n- reference_ID\n- lake_water_level\n\nThe *reference_ID* table contains the **Unique ID Number, Lake Name, Original Source, and Metadata**, of all lakes.\nThe **Unique ID Number** is assigned incrementally as new lakes are added to the GLLD. This is the *primary key* for the MySQL table. This number is what is primarly used to identify\nand reference lakes in the GLLD and LakePy. If the same lake exists from multiple sources, there will be two lakes with a unique ID.\nFor example, Lake Mead exists in both the hydroweb data and the USGS data. As shown below, both copies are retained and treated as unique entries.\n\n|    |   id_No | source   | lake_name                           |\n|---:|--------:|:---------|:------------------------------------|\n|  0 |     138 | hydroweb | Mead                                |\n|  1 |    1556 | usgs     | MEAD LAKE WEST BAY NEAR WILLARD, WI |\n\nThe **lake name** is copied exactly as is from the original data source. The **source** identifies which of the three original data sources\nthe data comes from. Finally the **metadata** stores all available metadata from the data source as a JSON array.\n\nThe **lake_water_level** table contains the actual historic data for each lake. This table uses a *composite primary key* \nof the **Unique ID Number** and **date (YYYY-MM-DD)** for the MYSQL table. This method of a composite primary key limits the GLLD to have a maximum resolution of\none unique lake measurement per day for each lake.\n\n## Functionality\nThe GLLD is regularly updated for new data (both the addition of new lakes, and the addition of new lake level data for existing lakes). \nThe precise way we interact with each of the original sources varies from source to source, but the logic of these interactions is consistent across all sources.\nThis occurs in three steps for each of the two tables:\n- Read\n- Verify\n- Commit\n\nfor the **Reference_ID** table, we point to summary product tables from each of the original data sources. Lake names are *read* in, and the\nassociated metadata is packaged as a JSON object. These are then *verified* against the existing GLLD. New lakes are assigned\na **Unique ID Number** and *committed* to the GLLD. Already existing Lakes are dropped to ensure no duplication of products within the GLLD.\n\nThe **lake_water_level** table operates similarly. The actual lake level data for each lake is *read* in and parsed to contain the data and lake level measurment.\nEach record is *verified* against the existing table, and all new data entries are *committed* to the GLLD. \n\n## Software Used\n**Built with:**\n- [Python](https://www.python.org/)\n  - [Pandas](https://pandas.pydata.org/)\n  - [PyMySQL](https://pymysql.readthedocs.io/en/latest/)\n  - [SQLAlchemy](https://www.sqlalchemy.org/)\n- [Amazon MySQL RDS](https://aws.amazon.com/rds/mysql/)\n\n## Credits\nThis work is based on funding provided by the ESIP Lab with support from the National Aeronautics and Space\nAdministration (NASA), National Oceanic and Atmospheric Administration (NOAA) and the United States Geologic\nSurvey (USGS). \n  \nThe authors would also like to acknowledge the Dynamic\nStratigraphy Group at UT Austin and the AWS\nStaff that helped on this project!\n\n## License\n\nCreative Commons © [James Hooker Gearon \u0026 John Franey](https://github.com/ESIPFed/GlobalLakeLevelDatabase/blob/master/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fesipfed%2Fglobal-lake-level-database","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fesipfed%2Fglobal-lake-level-database","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fesipfed%2Fglobal-lake-level-database/lists"}