{"id":15760613,"url":"https://github.com/flrs/caiso_wind_forecast","last_synced_at":"2025-05-07T10:49:12.787Z","repository":{"id":201981365,"uuid":"334811663","full_name":"flrs/caiso_wind_forecast","owner":"flrs","description":"Using Machine Learning and R to Forecast Wind Energy in the California Power Grid ⚡💨📈","archived":false,"fork":false,"pushed_at":"2024-04-09T21:32:28.000Z","size":15272,"stargazers_count":10,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-06T06:52:51.110Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flrs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-01T02:48:14.000Z","updated_at":"2024-10-03T20:10:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"490ba7a3-3dbc-4f16-957d-cad0c18eee03","html_url":"https://github.com/flrs/caiso_wind_forecast","commit_stats":null,"previous_names":["flrs/caiso_wind_forecast"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flrs%2Fcaiso_wind_forecast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flrs%2Fcaiso_wind_forecast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flrs%2Fcaiso_wind_forecast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flrs%2Fcaiso_wind_forecast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flrs","download_url":"https://codeload.github.com/flrs/caiso_wind_forecast/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252863396,"owners_count":21816023,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T10:59:16.267Z","updated_at":"2025-05-07T10:49:12.759Z","avatar_url":"https://github.com/flrs.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CAISO Wind Energy Forecast\n\n\u003cp align=\"center\"\u003e\n\u003cimg width=\"500\" alt=\"time series plot with labels\" src=\"./docs/example_plot_with_labels.png\"\u003e\n\u003c/p\u003e\n\nThis repository is about using machine learning to forecast the amount of wind energy in the California electricity \ngrid. With regards to the typical data science workflow, the repository covers modeling and limited feature engineering.\n\nThis project is executed in [R](https://www.r-project.org/), using the \n[Modeltime](https://business-science.github.io/modeltime/) library for modeling, and [MLFlow](https://mlflow.org/) for \ntracking experiments.\n\nThe motivation for this project is to explore the capabilities (and limits) of the [Modeltime](https://business-science.github.io/modeltime/)\ntime series machine learning library and to get a feeling for the challenges of modelling a part of the electricity grid.\n\n## Overview\n\nThe picture below gives an overview of the ETL and modeling process. To aid reproducibility, alphanumeric strings in \nlabels (e.g. `b2caff6`) refer to specific git commits on which these models were generated.\n\n![Model Architecture](./docs/model_architecture.svg)\n\n### About the Data\n\nAll data have been acquired and aggregated by the author from public sources. Three datasets are available to aid the \nforecasting effort:\n\n- [`db_pull_production_data_raw_20201208.csv`](./data/raw/db_pull_production_data_raw_20201208.csv) – 5-min time series data stating energy in the California grid by energy \n  source, according to [CAISO](http://www.caiso.com/TodaysOutlook/Pages/supply.aspx). Columns `Time` and `Wind` are\n  relevant for this analysis. `Wind` is what we are trying to forecast.\n  \n- [`db_pull_weather_data_raw_20201208.csv`](./data/raw/db_pull_weather_data_raw_20201208.csv) – 1-hour weather time series data at 10 key locations for renewable energy\n  generation (wind, solar) in California. The author determined these locations based on geospatial analysis of \n  renewable energy assets in the state, an analysis that is outside the scope of this repository. Columns starting with\n  `0_` through `4_` belong to wind-generating locations, ordered in descending order of generating capacity. This \n  dataset can be used to create features for the model. Weather data have been acquired from \n  [Dark Sky](https://darksky.net).\n  \n- [`db_pull_feature_gross_production_20201209.csv`](./data/raw/db_pull_feature_gross_production_20201209.csv) - 1-hour time series dataset with domain-informed features at 10 key\n  locations (see above). The author generated the wind-related features (`0_wind` through `4_wind`) by combining weather\n  and turbine power curve information. From the weather information above, the author generated air density and hub\n  height adjusted estimates of the available wind energy. That information was fed through an assumed power curve and\n  multiplied by the assumed total capacity available at that key location. That analysis is outside the scope of this \n  repository, but the generated features can be used for modelling.\n\nAll data are fed through a [feature engineering pipeline](./src/data/comb_data_rev8.R), for which relevant features have\nbeen selected using a [random forest model](./src/feature_engineering/rf_feat.R).\n\n### About the Model\n\nThe model is a weighted ensemble model of 8 tree-based models. Two of those models are based on \n[Cubist](https://cran.r-project.org/package=Cubist), a boosted regression model (find a great presentation about Cubist \nby [Max Kuhn](https://github.com/topepo) [here](https://static1.squarespace.com/static/51156277e4b0b8b2ffe11c00/t/56e3056a3c44d8779a61988a/1457718645593/cubist_BRUG.pdf)). \nThree models are based on XGBoost and another three models are random forest models. These types of\nmodels have been chosen according to their ability to incorporate a set of 188 features and good training performance\nin R.\n\nEach model has been trained on about 143k data points. The model parameters have been selected after hyperparameter\ntuning, subject to 4-fold time series cross-validation. The number of folds was constrained by training performance.\n\nA weighted average was chosen for ensembling the models due to performance constraints over potentially more accurate\nmethods like stacking. The weights were chosen after assessing 20 different weight combinations through a latin\nhypercube experimental design (find assessment [here](./util/etc/lhc_ensemble_results.csv)).\n\nIn summary, the modeling process was heavily constrained by performance considerations and available project time.\n\nAll model runs, including training and hyperparameter optimization, have been recorded in MLFlow. You can boot up a\nworkable MLFlow instance using the associated [Dockerfile](./util/mlflow/Dockerfile). All results have also been \nexported as CSVs in the [\"static\" directory](./util/mlflow/static), where every file name corresponds to an MLFlow \nexperiment.\n\n### Repository Structure\n\n- [data](./data) – Data used or produced in the modeling process\n  - [raw](./data/raw) – Raw data (see [above](#about-the-data))\n  - [processed](./data/processed) – Processed data\n- [docs](./docs) – Documentation\n- models – Trained models (removed to save storage space)\n  - ens_level1_f9e6c40.rds – Final, trained ensemble model (see [above](#about-the-model))\n- [notebooks](./notebooks) – Notebooks for experiments and analyses\n  - [ext](./notebooks/ext) – Notebooks used in external environments, for example for large-scale training in the cloud\n  - [analyze_performance.rmd](./notebooks/analyze_performance.rmd) ([HTML](./notebooks/analyze_performance.html) | [view in browser](https://flrs.github.io/caiso_wind_forecast/)) – Analysis of model performance\n- [src](./src) – Source code for building models\n  - [data](./src/data) – Source code for ETL process\n  - [feature_engineering](./src/feature_engineering)\n  - [models_hyperparam_tuning](./src/models_hyperparam_tuning)\n  - [models_training](./src/models_training)\n  - [util](./src/util) – Utility code related to building models\n- [util](./util) – Utility tools\n  - [mlflow](./util/mlflow) – Data related to MLFlow\n    - [mlruns](./util/mlflow/mlruns) – MLFlow data directory\n    - [static](./util/mlflow/static) – Static export of tracked MLFlow experiments\n    - [Dockerfile](./util/mlflow/Dockerfile) – Dockerfile for running MLFlow server\n    - [environment.yml](./util/mlflow/environment.yml) – Conda environment file to create MLFlow environment\n  \n## Model Performance\n\nThis section summarizes model performance by use case.\n\n### Time Series Forecasting Use Case\n\nThe table below shows the performance of the models at time series forecasting. Given the training and test set, it\nis clear that the Cubist models heavily overfit and the random forest models overfit to some extent as well.\n\nCross-validation (CV) has not been performed for the ensemble model due to performance constraints.\n\n| Model Name               | CV, folds | mae (train) | rmse (train) | rsq (train) | mae (test) | rmse (test) | rsq (test) | ratio mae train/test |\n|--------------------------|:---------:|:-----------:|:------------:|:-----------:|:----------:|:-----------:|:----------:|:--------------------:|\n| ens_level1_f9e6c40       | ❌        |             |              |             | 469        | 599         | 0.779      |                      |\n| cubist_level0_b2caff6_1  | ✅ 4      | 15          | 32           | 0.999       | 493        | 634         | 0.686      | 0.03                 |\n| cubist_level0_b2caff6_3  | ✅ 4      | 20          | 39           | 0.999       | 507        | 656         | 0.670      | 0.04                 |\n| rf_level0_cc50409_1      | ✅ 4      | 219         | 300          | 0.943       | 439        | 559         | 0.755      | 0.50                 |\n| rf_level0_cc50409_2      | ✅ 4      | 231         | 315          | 0.937       | 440        | 560         | 0.753      | 0.53                 |\n| rf_level0_cc50409_3      | ✅ 4      | 37          | 57           | 0.998       | 444        | 567         | 0.751      | 0.08                 |\n| xgboost_level0_9dc6cbe_1 | ✅ 4      | 580         | 772          | 0.759       | 640        | 799         | 0.678      | 0.91                 |\n| xgboost_level0_9dc6cbe_2 | ✅ 4      | 519         | 689          | 0.803       | 598        | 752         | 0.706      | 0.87                 |\n| xgboost_level0_9dc6cbe_3 | ✅ 4      | 565         | 752          | 0.763       | 628        | 783         | 0.670      | 0.90                 |\n\n### Time Series Peak Forecasting Use Case\n\nForecasting the peaks of the wind energy time series in the California grid is a specific use case of this model. \nThe [analyze_performance.rmd](./notebooks/analyze_performance.rmd) ([HTML](./notebooks/analyze_performance.html) | [view in browser](https://flrs.github.io/caiso_wind_forecast/)) \nnotebook investigates this case in detail. In summary, the peaks predicted by the model, in 75% of all cases,\ndo not miss the energy in the actual peaks within a 26-hour window by more than 14%.\n\n## Contribution\n\nI am happy about any contribution or feedback. Please let me know about your comments via the \n[Issues tab](https://github.com/flrs/caiso_wind_forecast/issues) on GitHub.\n\n## License and Attributions\n\nThis project is released under the [MIT License](./LICENSE).\n\nPlease note that raw data as provided in \n[`db_pull_production_data_raw_20201208.csv`](./data/raw/db_pull_production_data_raw_20201208.csv) have been generated\nby the [California ISO](http://www.caiso.com).\n\nPlease also note that weather data as provided in \n[`db_pull_weather_data_raw_20201208.csv`](./data/raw/db_pull_weather_data_raw_20201208.csv) has been extracted from \n[DarkSky](https://darksky.net) and is subject to its [Terms of Use](https://darksky.net/tos), allowing use only for \n\"personal, non-commercial purposes\".\n\nFor the [social preview picture](./docs/social_picture.png), the California bear is from \n[Vecteezy.com](https://www.vecteezy.com/free-vector/california-bear). The R Logo is used in its original form under the \n[CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/), and is (C) 2016 by The R Foundation. The Modeltime logo\nis taken from the [Modeltime GitHub repository](https://github.com/business-science/modeltime/blob/37c8bf2e177e1094043986a9bb7d9ae9a8a70da1/man/figures/logo.png)\nand is subject to the [MIT License](https://github.com/business-science/modeltime/blob/37c8bf2e177e1094043986a9bb7d9ae9a8a70da1/LICENSE.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflrs%2Fcaiso_wind_forecast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflrs%2Fcaiso_wind_forecast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflrs%2Fcaiso_wind_forecast/lists"}