{"id":20200829,"url":"https://github.com/openclimatefix/pvconsumer","last_synced_at":"2025-04-10T11:22:13.333Z","repository":{"id":37239599,"uuid":"460756134","full_name":"openclimatefix/PVConsumer","owner":"openclimatefix","description":"Consumer PV data from various sources","archived":false,"fork":false,"pushed_at":"2025-04-07T17:29:53.000Z","size":865,"stargazers_count":3,"open_issues_count":6,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-07T18:37:52.951Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openclimatefix.png","metadata":{"funding":{"github":["openclimatefix"],"patreon":null,"open_collective":"openclimatefix","ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null},"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-18T07:40:49.000Z","updated_at":"2024-12-04T14:58:34.000Z","dependencies_parsed_at":"2023-12-11T18:44:39.511Z","dependency_job_id":"6d02c33b-f501-46e5-8170-2f54e8c915e7","html_url":"https://github.com/openclimatefix/PVConsumer","commit_stats":{"total_commits":299,"total_committers":7,"mean_commits":"42.714285714285715","dds":0.4581939799331104,"last_synced_commit":"a0482d11525f5a4a966f7f24e291e158fd2b5307"},"previous_names":[],"tags_count":84,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVConsumer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVConsumer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVConsumer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2FPVConsumer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openclimatefix","download_url":"https://codeload.github.com/openclimatefix/PVConsumer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248208604,"owners_count":21065203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T04:45:47.568Z","updated_at":"2025-04-10T11:22:13.302Z","avatar_url":"https://github.com/openclimatefix.png","language":"Python","funding_links":["https://github.com/sponsors/openclimatefix","https://opencollective.com/openclimatefix"],"categories":[],"sub_categories":[],"readme":"# PVConsumer\n\n\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n[![All Contributors](https://img.shields.io/badge/all_contributors-3-orange.svg?style=flat-square)](#contributors-)\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n\n[![codecov](https://codecov.io/gh/openclimatefix/PVConsumer/branch/main/graph/badge.svg?token=R0VM4YBUJS)](https://codecov.io/gh/openclimatefix/PVConsumer)\n\nConsumer PV data from `pvoutput.org` and `SheffieldSolar`. The idea is to expand to different data sources in the future.\n\n\n# Live\n\nThis application pulls live data from PV output.org and stores it in a database\n\n## Dependencies\n\n* [poetry][poetry]\n\n## Run locally\n\nTo run the application locally\n\n```bash\n# Install the python dependencies\npoetry install\n\npoetry run pvconsumer/app.py\n```\n\n## Logic\n\nThe app has the following high-level strucuture\n```mermaid\n  graph TD;\n      A[1. Get PV system]--\u003eB;\n      B[2. Filter PV Systems]--\u003eC;\n      C[3. Pull Data]--\u003eD[4. Save data];\n```\n\n1. Get PV System\n```mermaid\n   graph TD\n    A0(Start) --\u003e A1\n    A0(Start) --\u003e A2\n    A1[Load local PV systems] --\u003e A3{Are all PV system in the database}\n    A2[Load Database PV systems] --\u003e A3\n    A3 --\u003e |No| A4[Load the extra \u003cbr/\u003e PV systems from pvoutput.org]\n    A3 --\u003e |yes| A5(Finish)\n    A4 --\u003e A5\n```\n\n2. Filter PV Systems\n```mermaid\n   graph TD\n    B0(Start) --\u003e B1{Is there any PV data in \u003cbr/\u003e our database for this PV system?}\n    B1 --\u003e |No| B2[Keep PV system]\n    B1--\u003e |yes| B3{Is there any more PV data, \u003cbr/\u003e from pv output.org, \u003cbr/\u003eavailable for this PV system?}\n    B3 --\u003e |yes| B2\n    B3 --\u003e |No| B5[Dischagre PV system]\n    B2 --\u003e B6(Finish)\n    B5 --\u003e B6\n```\n3. Pull Data\n```mermaid\n   graph TD\n    C0(Start) --\u003e C1[Pull Data from pvoutput.prg]\n    C1 --\u003e C2{Is this data \u003cbr/\u003e in our database already?}\n    C2 --\u003e |yes| C3[Keep PV data]\n    C2 --\u003e |No| C4[Dischagre PV data]\n    C3 --\u003e C5(Finish)\n    C4 --\u003e C5\n\n```\n\n## Code style\n\nFormat the python codebase **in place**.\n\n```bash\nmake format\n```\n\nTest that the codebase is formatted\n\n```bash\nmake lint\n```\n\n\n## 🩺 Testing\n\nRun only the unit tests\n\n```bash\npoetry run pytest tests/unittest\n```\n\nRun all the tests (including the \"integration\" tests that require credentials to call an external API)\n\n```bash\npoetry run pytest tests\n```\n\n## Environmental Variables\n\n- API_KEY: API key for pvoutput.org\n- SYSTEM_ID: System id for pvoutput.org\n- DATA_SERVICE_URL: data service url for pvoutput.org\n- SS_URL: Sheffield Solar URL\n- SS_API_KEY: Sheffield Solar API key\n- SS_SYSTEM_ID: Sheffield Solar System ID\n- DB_URL: Save in database to the pv database\n\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"http://lostcoding.com\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/20285369?v=4?s=100\" width=\"100px;\" alt=\"Brandon Ly\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eBrandon Ly\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/PVConsumer/commits?author=branberry\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"http://lefun.fun\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/1105380?v=4?s=100\" width=\"100px;\" alt=\"Simon Lemieux\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSimon Lemieux\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/PVConsumer/commits?author=simlmx\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/peterdudfield\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/34686298?v=4?s=100\" width=\"100px;\" alt=\"Peter Dudfield\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003ePeter Dudfield\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/PVConsumer/commits?author=peterdudfield\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n\n[poetry]: https://python-poetry.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fpvconsumer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenclimatefix%2Fpvconsumer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fpvconsumer/lists"}