{"id":15064067,"url":"https://github.com/drkenreid/global-temperature-analysis","last_synced_at":"2026-01-20T04:05:49.383Z","repository":{"id":254762214,"uuid":"847128872","full_name":"DrKenReid/global-temperature-analysis","owner":"DrKenReid","description":"A work-in-progress data analysis pipeline that leverages R, SQL Server, and Tableau to download, process, clean, and analyze global temperature data, aiming to uncover long-term climate trends and patterns.","archived":false,"fork":false,"pushed_at":"2024-09-22T03:52:58.000Z","size":65399,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-21T17:06:48.106Z","etag":null,"topics":["climate","climate-change","climate-data","climate-model","climate-science","data-science","msql","msql-database","r","sql","tsql"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DrKenReid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-24T23:50:22.000Z","updated_at":"2024-09-22T03:53:01.000Z","dependencies_parsed_at":"2024-10-13T00:02:51.863Z","dependency_job_id":"f0726243-2bbc-42b5-acaf-b5172216ae92","html_url":"https://github.com/DrKenReid/global-temperature-analysis","commit_stats":null,"previous_names":["drkenreid/global-temperature-analysis"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DrKenReid%2Fglobal-temperature-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DrKenReid%2Fglobal-temperature-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DrKenReid%2Fglobal-temperature-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DrKenReid%2Fglobal-temperature-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DrKenReid","download_url":"https://codeload.github.com/DrKenReid/global-temperature-analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225769465,"owners_count":17521258,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["climate","climate-change","climate-data","climate-model","climate-science","data-science","msql","msql-database","r","sql","tsql"],"created_at":"2024-09-25T00:10:59.084Z","updated_at":"2026-01-20T04:05:49.368Z","avatar_url":"https://github.com/DrKenReid.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌡️ Global Temperature Analysis Project 🌍\n\nA comprehensive data analysis pipeline for NOAA global temperature data, using R and SQL Server.\n\n## 📋 Data Source\n\nUses NOAAGlobalTemp dataset, including:\n\n- Global Historical Climate Network-Monthly (GHCNm) for land\n- Extended Reconstructed Sea Surface Temperature (ERSST) for sea\n- ICOADS and IABP for Arctic Ocean\n\n## 📊 Dataset Details\n\n1. **Time Series Data**: `combined_time_series.csv` (1850-present, anomalies vs. 1901-2000 average)\n2. **Gridded Data**: `gridded_data.csv` (5° x 5° grid, anomalies vs. 1991-2020 base)\n\n## 👥 Who Is This For?\n\n- Climate researchers\n- Data scientists working with environmental data\n- Anyone interested in global temperature patterns\n\n## ✨ Features\n\n- Automated data download and processing of raw NOAA temperature data\n- Robust SQL database for data storage and querying\n- Comprehensive data cleaning and analysis\n- Calculates statistics and identifies temperature trends\n- Exports results for further use\n- Enhanced error handling and detailed logging\n- Improved data consistency checks\n- Modular SQL script execution\n- Automated database setup and table creation\n- Progress tracking for data conversion and processing\n\n## 🛠️ Requirements\n\n- **R** (3.6.0+)\n- **SQL Server** (2019+)\n- **R packages**:\n\n  ```\n  curl, DBI, dplyr, httr, ncdf4, odbc, readr, xml2, progress, lubridate, tidyverse\n  ```\n\n  The script will automatically install and load these packages if they are not already available in your R environment.\n\n## 🚀 How to Use\n\n1. **Ensure** you have R and SQL Server installed on your system.\n2. **Clone** the repository.\n3. **Open** R or RStudio and set the working directory to the `R/` folder.\n4. **Run** `runner.R`.\n5. The script will automatically install any missing packages.\n6. **Check** `data/processed/` for results and the SQL database for exploration data.\n\nThe script handles package installation, database setup, data download, and processing automatically.\n\n## 📁 Project Structure\n\nBefore running, your project should look like this. After running, you'll find the empty folders filled with downloaded raw temperature data, processed CSVs, and more.\n\n```\ntemperature-analysis-project/\n│\n├── data/\n│   ├── raw/\n│   └── processed/\n│\n├── docs/\n│   ├── database_schema.md\n│   ├── data_dictionary.md\n│   └── data_processing_pipeline.md\n│\n├── outputs/\n│   ├── plots/\n│   ├── tableau/\n│   └── tables/\n│\n├── R/\n│   ├── runner.R\n│   └── utils.R\n│\n├── sql/\n│   ├── drop_tables.sql\n│   ├── explore_data.sql\n│   ├── process_data.sql\n│   ├── run_diagnostics.sql\n│   └── setup_database.sql\n│\n├── .gitignore\n└── README.md\n```\n\n## 🔍 Key Features\n\n- Automated data download and conversion\n- SQL database creation and management\n- Enhanced error handling and logging in R scripts\n- Improved SQL script execution with support for multiple statements\n- Automated database and table creation\n- Data consistency checks for `TimeSeries` and `GriddedData` tables\n- Detailed diagnostic queries for data verification\n- Progress bars for data conversion and processing tasks\n\n## 🔜 Upcoming Features\n\n- Advanced statistical analysis\n- Machine learning integration\n- Interactive visualization dashboard\n- Geospatial analysis\n- Correlation with other climate indicators\n\n## 🤝 Contributions\n\nContributions, bug reports, and feature requests are welcome!\n\n## 📜 Data Use and Citation\n\nWhen using this data, please cite:\n\n\u003e NOAA National Centers for Environmental Information, Climate at a Glance: Global Time Series, published [Month] 2024, retrieved on [Date] from https://www.ncei.noaa.gov/access/monitoring/climate-at-a-glance/global/time-series\n\n**Note**: Replace `[Month]` and `[Date]` with the actual month and date of retrieval.\n\n## 📄 License\n\nMIT License\n\n---","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdrkenreid%2Fglobal-temperature-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdrkenreid%2Fglobal-temperature-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdrkenreid%2Fglobal-temperature-analysis/lists"}