{"id":13949189,"url":"https://github.com/ECMWFCode4Earth/ml_drought","last_synced_at":"2025-07-20T11:31:08.491Z","repository":{"id":67984560,"uuid":"184443121","full_name":"ECMWFCode4Earth/ml_drought","owner":"ECMWFCode4Earth","description":"Machine learning to better predict and understand drought. Moving  github.com/ml-clim","archived":false,"fork":false,"pushed_at":"2022-05-18T18:41:07.000Z","size":323678,"stargazers_count":93,"open_issues_count":42,"forks_count":19,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-07-10T04:01:31.463Z","etag":null,"topics":["2019","copernicus","machine-learning"],"latest_commit_sha":null,"homepage":"https://ml-clim.github.io/drought-prediction/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ECMWFCode4Earth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-05-01T16:05:56.000Z","updated_at":"2024-11-18T19:31:29.000Z","dependencies_parsed_at":"2023-09-19T01:25:51.377Z","dependency_job_id":null,"html_url":"https://github.com/ECMWFCode4Earth/ml_drought","commit_stats":{"total_commits":248,"total_committers":3,"mean_commits":82.66666666666667,"dds":"0.45967741935483875","last_synced_commit":"f690055177599493ac1633cad27a26705ce225c4"},"previous_names":["esowc/ml_drought"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ECMWFCode4Earth/ml_drought","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ECMWFCode4Earth%2Fml_drought","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ECMWFCode4Earth%2Fml_drought/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ECMWFCode4Earth%2Fml_drought/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ECMWFCode4Earth%2Fml_drought/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ECMWFCode4Earth","download_url":"https://codeload.github.com/ECMWFCode4Earth/ml_drought/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ECMWFCode4Earth%2Fml_drought/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265563652,"owners_count":23788740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2019","copernicus","machine-learning"],"created_at":"2024-08-08T05:01:40.254Z","updated_at":"2025-07-20T11:31:08.475Z","avatar_url":"https://github.com/ECMWFCode4Earth.png","language":"Jupyter Notebook","funding_links":[],"categories":["Natural Resources"],"sub_categories":["Soil and Land"],"readme":"\u003c!-- ![](https://imgur.com/8qjbXcD) --\u003e\n[![Build Status](https://travis-ci.com/esowc/ml_drought.svg?branch=master)](https://travis-ci.com/esowc/ml_drought)\n\n\u003c!-- https://colab.research.google.com/github/googlecolab/colabtools/blob/master/notebooks/colab-github-demo.ipynb#scrollTo=8QAWNjizy_3O --\u003e\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/esowc/ml_drought/blob/master/notebooks/docs/Pipeline.ipynb)\n\n\u003c!-- OVERVIEW --\u003e\n\n# A Machine Learning Pipeline for Climate Science\n\nThis repository is an end-to-end pipeline for the creation, intercomparison and evaluation of machine learning methods in climate science.\n\nThe pipeline carries out a number of tasks to create a unified-data format for training and testing machine learning methods.\n\nThese tasks are split into the different classes defined in the `src` folder and explained further below:\n\n\u003cimg src=\"https://github.com/esowc/ml_drought/blob/master/img/pipeline_overview.png\" width=\"600\"  style=\"margin:auto; width:70%; padding:10px;\"\u003e\n\nNOTE: some basic working knowledge of Python is required to use this pipeline, although it is not too onerous\n\n\u003c!-- HOW TO USE THE PIPELINE --\u003e\n\n## Using the Pipeline \u003ca name=\"using\"\u003e\u003c/a\u003e\n\nThere are three entrypoints to the pipeline:\n* [run.py](run.py)\n* [notebooks](notebooks/docs)\n* [scripts](scripts/README.md)\n\nA blog post describing the goals and design of the pipeline can be found\n[here](https://medium.com/@gabrieltseng/a-machine-learning-pipeline-for-climate-research-ebf83b2b349a).\n\nView the initial presentation of our pipeline [here](https://www.youtube.com/watch?v=QVFiGERCiYs).\n\n## Setup \u003ca name=\"setup\"\u003e\u003c/a\u003e\n\n[Anaconda](https://www.anaconda.com/download/#macos) running python 3.7 is used as the package manager. To get set up\nwith an environment, install Anaconda from the link above, and (from this directory) run\n\n```bash\nconda env create -f environment.yml\n```\nThis will create an environment named `esowc-drought` with all the necessary packages to run the code. To\nactivate this environment, run\n\n```bash\nconda activate esowc-drought\n```\n\n[Docker](https://www.docker.com/) can also be used to run this code. To do this, first\nrun the docker app (either [docker desktop](https://www.docker.com/products/docker-desktop))\nor configure the `docker-machine`:\n\n```bash\n# on macOS\nbrew install docker-machine docker\n\ndocker-machine create --driver virtualbox default\ndocker-machine env default\n```\nSee [here](https://stackoverflow.com/a/33596140/9940782) for help on all machines or [here](https://stackoverflow.com/a/49719638/9940782)\nfor MacOS.\n\n\nThen build the docker image:\n\n```bash\ndocker build -t ml_drought .\n```\n\nThen, use it to run a container, mounting the data folder to the container:\n\n```bash\ndocker run -it \\\n--mount type=bind,source=\u003cPATH_TO_DATA\u003e,target=/ml_drought/data \\\nml_drought /bin/bash\n```\n\nYou will also need to create a .cdsapirc file with the following information:\n```bash\nurl: https://cds.climate.copernicus.eu/api/v2\nkey: \u003cINSERT KEY HERE\u003e\nverify: 1\n```\n\n### Testing  \u003ca name=\"testing\"\u003e\u003c/a\u003e\n\nThis pipeline can be tested by running `pytest`. [flake8](http://flake8.pycqa.org) is used for linting.\n\nWe use [mypy](https://github.com/python/mypy) for type checking. This can be run by running `mypy src` (this runs mypy on the `src` directory).\n\nWe use [black](https://black.readthedocs.io/en/stable/) for code formatting.\n\n\u003c!-- PROJECT TEAM MEMBERS --\u003e\n\n__Team:__ [@tommylees112](https://github.com/tommylees112), [@gabrieltseng](https://github.com/gabrieltseng)\n\nFor updates follow [@tommylees112](https://twitter.com/tommylees112) on twitter or look out for our blog posts!\n\n- [Blog 1: Great News!](https://tommylees112.github.io/posts/2019/1/esowc_kick_off)\n- [Blog 2: The Pipeline](https://medium.com/@gabrieltseng/a-machine-learning-pipeline-for-climate-research-ebf83b2b349a)\n- [Blog 3: The Close of the Project!](https://tommylees112.github.io/posts/2019/2/esowc_final)\n\n\u003c!-- ESoWC initial outline --\u003e\n\n## Acknowledgements \u003ca name=\"acknowledgements\"\u003e\u003c/a\u003e\nThis was a project completed as part of the ECMWF Summer of Weather Code [Challenge #12](https://github.com/esowc/challenges_2019/issues/14). The challenge was setup to use [ECMWF/Copernicus open datasets](https://cds.climate.copernicus.eu/#!/home) to evaluate machine learning techniques for the **prediction of droughts**.\n\nHuge thanks to @ECMWF for making this project possible!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FECMWFCode4Earth%2Fml_drought","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FECMWFCode4Earth%2Fml_drought","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FECMWFCode4Earth%2Fml_drought/lists"}