{"id":23116313,"url":"https://github.com/nabilshadman/python-uk-weather-analytics","last_synced_at":"2026-04-13T00:44:17.026Z","repository":{"id":182022392,"uuid":"378307883","full_name":"nabilshadman/python-uk-weather-analytics","owner":"nabilshadman","description":"An end-to-end data science workflow of UK weather data","archived":false,"fork":false,"pushed_at":"2024-12-03T14:32:35.000Z","size":3433,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-03T15:31:43.082Z","etag":null,"topics":["data-engineering","data-science","data-visualization","machine-learning","matplotlib","numpy","pandas","scikit-learn"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nabilshadman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-19T02:40:10.000Z","updated_at":"2024-12-03T14:41:11.000Z","dependencies_parsed_at":"2024-12-03T15:24:50.204Z","dependency_job_id":"3f46532e-6da8-42d4-998f-37cf2f7adcf9","html_url":"https://github.com/nabilshadman/python-uk-weather-analytics","commit_stats":null,"previous_names":["nabilshadman/python-uk-weather-analytics"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fpython-uk-weather-analytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fpython-uk-weather-analytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fpython-uk-weather-analytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fpython-uk-weather-analytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nabilshadman","download_url":"https://codeload.github.com/nabilshadman/python-uk-weather-analytics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230061431,"owners_count":18166705,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-science","data-visualization","machine-learning","matplotlib","numpy","pandas","scikit-learn"],"created_at":"2024-12-17T04:15:58.604Z","updated_at":"2026-04-13T00:44:16.965Z","avatar_url":"https://github.com/nabilshadman.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UK Weather Analytics\n\n[![MIT License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/Python-3.7%2B-blue)](https://www.python.org/)\n[![Jupyter](https://img.shields.io/badge/Jupyter-Notebook-orange)](https://jupyter.org/)\n\n## Overview\n\nThis [research](https://github.com/nabilshadman/python-uk-weather-analytics/blob/main/report/uk_weather_analytics_report.pdf) project analyzes United Kingdom (UK) weather patterns using machine learning approaches, combining both unsupervised and supervised learning algorithms. Our analysis:\n\n- Identifies distinct regional weather station clusters based on natural variations in weather patterns\n- Develops classification models to predict station regions with high accuracy\n- Investigates potential correlations between weather conditions and happiness metrics in the UK\n\nThe complete analysis workflow is automated and reproducible, with all code and data publicly available.\n\n## Tech Stack\n\n- **Core Language:** Python 3.7+\n- **Key Libraries:**\n  - NumPy: Scientific computing and array operations\n  - Pandas: Data manipulation and analysis\n  - Matplotlib: Data visualization\n  - scikit-learn: Machine learning algorithms\n  - Jupyter: Interactive notebook environment\n- **Scripting:** Bash\n\n## Datasets\n\nOur analysis leverages two primary public datasets:\n\n1. **UK Historic Weather Data** (Met Office)\n   - Comprehensive monthly weather station records\n   - Geographical distribution shown in Figure 1 below\n\n2. **UK Personal Well-being Estimates** (Office for National Statistics)\n   - Annual Population Survey data (April 2014 - March 2015)\n   - Includes geographical breakdown\n\n![map_weather_stations_60](https://github.com/nabilshadman/python-uk-weather-analytics/assets/13073461/e289bcc0-6b45-4005-9b2d-b1e283160941)  \n**Figure 1:** Geographic distribution of UK weather stations\n\n### Feature Descriptions\n\n| Dataset | Feature | Definition |\n|---------|----------|------------|\n| Historic Station Data | tmax | Mean daily maximum temperature (°C) |\n| | tmin | Mean daily minimum temperature (°C) |\n| | af | Days of air frost |\n| | rain | Total rainfall (mm) |\n| | sun | Total sunshine duration (hours) |\n| Personal Well-being | average rating | Mean happiness rating (0-10 scale) |\n\n## Development Environment\n\n### Prerequisites\n\n- Anaconda Distribution (Recommended)\n- Python 3.7 or higher\n- Jupyter Notebook\n\n### Setup Instructions\n\n1. Install [Anaconda Distribution](https://www.anaconda.com/download)\n2. Launch Anaconda Navigator\n3. Start Jupyter Notebook via the \"Launch\" button\n4. Navigate to the project directory\n\n## Automated Workflow\n\nOur end-to-end automated pipeline ensures reproducibility and efficient data processing. The workflow consists of five main stages:\n\n### 1. Weather Data Acquisition\n```bash\ncode/test_automation/download_weather_data.ipynb\n```\n- Downloads station data based on `stations.txt` configuration\n- Creates cleaned text files without metadata\n\n### 2. Happiness Data Collection\n```bash\ncode/test_automation/download_happiness_data.ipynb\n```\n- Fetches personal well-being estimates\n- Stores data in Excel format\n\n### 3. Weather Clustering Analysis\n```bash\ncode/test_automation/perform_clustering_weather_data.ipynb\n```\n- Processes and cleans station data\n- Performs clustering analysis\n- Generates visualizations\n- Saves intermediate results for subsequent stages\n\n### 4. Regional Classification\n```bash\ncode/test_automation/perform_classification_weather_data.ipynb\n```\n- Builds classification models\n- Evaluates regional prediction accuracy\n- Produces performance metrics\n\n### 5. Weather-Happiness Regression\n```bash\ncode/test_automation/perform_regression_weather_happiness_datasets.ipynb\n```\n- Combines weather and well-being datasets\n- Conducts regression analysis\n- Generates statistical summaries\n\n![data_science_workflow](https://github.com/nabilshadman/python-uk-weather-analytics/assets/13073461/267c5459-6bd4-46d8-8f69-785e0a9e9f52)  \n**Figure 2:** Complete data science pipeline and file dependencies\n\n## Documentation\n\nFor detailed methodology, analysis, and conclusions, please refer to our comprehensive [research report](https://github.com/nabilshadman/python-uk-weather-analytics/blob/main/report/uk_weather_analytics_report.pdf).\n\n## Contributing\n\nWe welcome contributions! Please feel free to submit pull requests or open issues for any improvements.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](./LICENSE) file for details.\n\n## Citation\n\nIf you use this work in your research, please cite:\n\n```bibtex\n@misc{uk-weather-analytics,\n  author = {Shadman, Nabil},\n  title = {UK Weather Analytics},\n  year = {2021},\n  publisher = {GitHub},\n  url = {https://github.com/nabilshadman/python-uk-weather-analytics}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnabilshadman%2Fpython-uk-weather-analytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnabilshadman%2Fpython-uk-weather-analytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnabilshadman%2Fpython-uk-weather-analytics/lists"}