{"id":20200827,"url":"https://github.com/openclimatefix/pvnet-summation","last_synced_at":"2026-03-16T18:34:02.025Z","repository":{"id":182539084,"uuid":"667449391","full_name":"openclimatefix/PVNet-summation","owner":"openclimatefix","description":"This project is used for training a model to sum the GSP predictions of PVNet into a national estimate","archived":false,"fork":false,"pushed_at":"2025-06-02T17:20:56.000Z","size":3478,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-03T06:22:59.841Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openclimatefix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["openclimatefix"],"patreon":null,"open_collective":"openclimatefix","ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2023-07-17T14:25:37.000Z","updated_at":"2025-05-20T11:01:07.000Z","dependencies_parsed_at":"2023-08-14T15:50:36.668Z","dependency_job_id":"ec79193f-38a5-4dd2-938b-5e1b32de19b3","html_url":"https://github.com/openclimatefix/PVNet-summation","commit_stats":{"total_commits":57,"total_committers":5,"mean_commits":11.4,"dds":"0.42105263157894735","last_synced_commit":"940e8f094535b019165349d14ab4fe795029e263"},"previous_names":["openclimatefix/pvnet_summation","openclimatefix/pvnet-summation"],"tags_count":23,"template":false,"template_full_name":"openclimatefix/ocf-template","purl":"pkg:github/openclimatefix/PVNet-summation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVNet-summation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVNet-summation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVNet-summation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVNet-summation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openclimatefix","download_url":"https://codeload.github.com/openclimatefix/PVNet-summation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVNet-summation/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261289419,"owners_count":23136070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T04:45:46.305Z","updated_at":"2026-03-16T18:34:01.964Z","avatar_url":"https://github.com/openclimatefix.png","language":"Python","funding_links":["https://github.com/sponsors/openclimatefix","https://opencollective.com/openclimatefix"],"categories":[],"sub_categories":[],"readme":"# PVNet summation\n[![ease of contribution: hard](https://img.shields.io/badge/ease%20of%20contribution:%20hard-bb2629)](https://github.com/openclimatefix/ocf-meta-repo?tab=readme-ov-file#overview-of-ocfs-nowcasting-repositories)\n\nThis project is used for training a model to sum the GSP predictions of [PVNet](https://github.com/openclimatefix/PVNet) into a national estimate.\n\nUsing this model to sum the GSP predictions rather than doing a simple sum increases the accuracy of the national predictions and can be configured to produce estimates of the uncertainty range of the national estimate. See the [PVNet](https://github.com/openclimatefix/PVNet) repo for more details and our paper.\n\n\n## Setup / Installation\n\n```bash\ngit clone https://github.com/openclimatefix/PVNet_summation\ncd PVNet_summation\npip install .\n```\n\n### Additional development dependencies\n\n```bash\npip install \".[dev]\"\n```\n\n## Getting started with running PVNet summation\n\nIn order to run PVNet summation, we assume that you are already set up with\n[PVNet](https://github.com/openclimatefix/PVNet) and have met all the requirements there.\n\nBefore running any code, copy the example configuration to a\nconfigs directory:\n\n```\ncp -r configs.example configs\n```\n\nYou will be making local amendments to these configs.\n\n### Datasets\n\nThe datasets required are the same as documented in\n[PVNet](https://github.com/openclimatefix/PVNet). The only addition is that you will need PVLive\ndata for the national sum i.e. GSP ID 0.\n\n\n## Generating pre-made concurrent batches of data for PVNet\n\nIt is required that you preprepare batches using the `save_concurrent_batches.py` script from\nPVNet. This saves the batches as required by the PVNet model to make predictions for all GSPs for\na single forecast init time. Seen the PVNet package for more details on this.\n\n\n### Set up and config example for batch creation\n\n\nThe concurrent batches created in the step above will be augmented with a few additional pieces of\ndata required for the summation model. Within your copy of `PVNet_summation/configs` make sure you\nhave replaced all of the items marked with `PLACEHOLDER`\n\n### Training PVNet_summation\n\nHow PVNet_summation is run is determined by the extensive configuration in the config files. The\nconfigs stored in `PVNet/configs.example` should work with batches created using the steps and\nbatch creation config mentioned above.\n\nMake sure to update the following config files before training your model:\n\n1. In `configs/datamodule/default.yaml`:\n    - update `batch_dir` to point to the directory you stored your concurrent batches in during\n      batch creation.\n    - update `gsp_zarr_path` to point to the PVLive data containing the national estimate\n2. In `configs/model/default.yaml`:\n    - update the PVNet model for which you are training a summation model for. A new summation model\n      should be trained for each PVNet model\n    - update the hyperparameters and structure of the summation model\n3. In `configs/trainer/default.yaml`:\n    - set `accelerator: 0` if running on a system without a supported GPU\n4. In `configs.config.yaml`:\n    - It is recommended that you set `presave_pvnet_outputs` to `True`. This means that the\n      concurrent batches that you create will only be run through the PVNet model once before\n      training and their outputs saved, rather than being run on the fly on each batch throughout\n      training. This can speed up training significantly.\n\n\nAssuming you have updated the configs, you should now be able to run:\n\n```\npython run.py\n```\n\n\n## Testing\n\nYou can use `python -m pytest tests` to run tests\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fpvnet-summation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenclimatefix%2Fpvnet-summation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fpvnet-summation/lists"}