{"id":20200939,"url":"https://github.com/openclimatefix/analysis-dashboard","last_synced_at":"2025-06-19T17:35:54.394Z","repository":{"id":99304083,"uuid":"601598725","full_name":"openclimatefix/analysis-dashboard","owner":"openclimatefix","description":"This is a Streamlit app for the OCF team that reports database statistics","archived":false,"fork":false,"pushed_at":"2025-06-13T12:57:15.000Z","size":3278,"stargazers_count":5,"open_issues_count":21,"forks_count":12,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-13T13:52:12.439Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openclimatefix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["openclimatefix"],"patreon":null,"open_collective":"openclimatefix","ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2023-02-14T12:14:31.000Z","updated_at":"2025-06-13T12:57:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"fd87806f-95fd-4404-9383-c0674eb4f0f1","html_url":"https://github.com/openclimatefix/analysis-dashboard","commit_stats":{"total_commits":237,"total_committers":7,"mean_commits":"33.857142857142854","dds":0.6286919831223629,"last_synced_commit":"322b635932aa457ef2a308d29298137b25bd74fd"},"previous_names":["openclimatefix/uk-analysis-dashboard","openclimatefix/analysis-dashboard"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/openclimatefix/analysis-dashboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2Fanalysis-dashboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2Fanalysis-dashboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2Fanalysis-dashboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2Fanalysis-dashboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openclimatefix","download_url":"https://codeload.github.com/openclimatefix/analysis-dashboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openclimatefix%2Fanalysis-dashboard/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260797404,"owners_count":23064774,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T04:46:14.992Z","updated_at":"2025-06-19T17:35:49.381Z","avatar_url":"https://github.com/openclimatefix.png","language":"Python","funding_links":["https://github.com/sponsors/openclimatefix","https://opencollective.com/openclimatefix"],"categories":[],"sub_categories":[],"readme":"# uk analysis dashboard\n\n\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n\n[![All Contributors](https://img.shields.io/badge/all_contributors-8-orange.svg?style=flat-square)](#contributors-)\n\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n\n[![ease of contribution: hard](https://img.shields.io/badge/ease%20of%20contribution:%20hard-bb2629)](https://github.com/openclimatefix#how-easy-is-it-to-get-involved)\n\nInternal dashboard for OCF to track forecast statistics and log the historical data of the forecast performance.\n\nThe analysis dashboard is a tool that was developed for OCF’s internal use and continues to evolve.\n\nBuilt with [Streamlit](https://streamlit.io/), a Python-based framework made specifically for creating data apps, the dashboard tracks and displays Quartz Solar and other data model statistics, such as mean absolute error (MAE), normalized mean absolute error (nMAE) for all the client sites. The database provides the error statistic using comparing the live generation with the forecast provided. Internally it has the option of chosing the forecast horizion to check the performance with genration. The larger the error, the less accurate the forecast.\n\nThanks to the analysis dashboard, OCF has a valuable feedback tool for understanding the accuracy of the forecasts being provided to it's clients.\n\n## Installation\n\n### **Manual Installation**\n\nYou can install the `analysis-dashboard` package directly from GitHub.\n\nIn the main project folder, install requirements:\n\n```shell\npip install -r requirements.txt\n```\n\nRun `streamlit hello` to check that Streamlit installed. A \"Welcome to Streamlit!\" page should open in the browser.\n\nCreate a login secret: `\n\n```shell\necho \"password = example\" \u003e src/.streamlit/secrets.toml\n```\n\n## Database connection\n\nTo run the app locally, you'll need to connect it to the `forecast development database`\n\nOCF team members can connect to the `forecast development database` using [these Notion instructions](https://www.notion.so/openclimatefix/Connecting-to-AWS-RDS-bf35b3fbd61f40df9c974c240e042354). Add `DB_URL= (db_url from notion documents)` to a `secrets.toml` file. Follow the instructions in the Notion document to connect to the database v.\n\nRun app:\n\n```shell\ncd src \u0026\u0026 streamlit run main.py\n```\n\n---\n\n### **Using Docker Compose**\n\nThis method uses Docker Compose to set up the app and its environment automatically.\n\n**Prerequisites:**\n\nYou need to have Docker and Docker Compose installed on your machine. If you don't have them, you can download them from the [Docker website](https://www.docker.com/get-started).\n\n**Steps:**\n\n1. Clone the repository and navigate to the project folder:\n\n```shell\ngit clone https://github.com/openclimatefix/analysis-dashboard.git\n\ncd analysis-dashboard\n```\n\n2. Create a `.env` file in the root directory and add the following environment variables:\n\n```shell\n# DB_URL=your-database-url      # Optional, if not available, you can skip this line\nREGION=india                  # Choose 'india' or 'uk'\nENVIRONMENT=development       # or 'production'\npassword=example              # Set your password here\nSHOW_PVNET_GSP_SUM=0          # Set this to 1 if you want to show pvnet_gsp_sum model\n```\n\n3. Create a `secrets.toml` file in the `src/.streamlit` directory and add the following line:\n\n```shell\necho \"password = example\" \u003e src/.streamlit/secrets.toml\n```\n\n4. Build the Docker image and start the app:\n\n```shell\ndocker-compose up --build\n```\n\n5. Open your browser and go to `http://localhost:8501` to view the app.\n\n6. To stop the app, press `Ctrl+C` in the terminal, and then run:\n\n```shell\ndocker-compose down\n```\n\n## Files\n\n### main.py\n\n`main.py` contains functions for the `home page` of the app, which focuses on MAE for the OCF `Quartz Solar` forecast.\n\n### main_india.py\n\n`main_india.py` contains functions for the `home page` of the app, which focuses on MAE for the OCF `Quartz Energy` forecast.\n\n### forecast.py\n\n`forecast.py` contains functions for the `forecast page`. The forecast page looks at how well each of OCF's forecast models is performing compared to `PVLive updated` truth values.\n\n### status.py\n\n`status.py` contains functionality for the `status pagwe` and allows the OCF team to update the forecast status in the database. This is one of the advantages of using an interface like Streamlit, facilitating status updates in a database.\n\n### auth.py\n\n`auth.py` contains code for the basic authenticaion that's been put in place.\n\n### pvsite_forecast.py\n\n`pvsite_forecast.py` contains the formulas and the metrics used to calculate MAE, nMAE and penalty incured against all sites.\n\n### site_toolbox.py\n\n`site_toolbox.py` is a page on the dashboard that can be used to get details of any particular site that OCF provides forecast to.\n\n### plots/make_pinball_and_exceedance_plots.py\n\nFunction to make `pinball` and exceedance plots. This shows how good the probabilistic forecasts are doing.\n\n### plots/ramp_rate.py\n\nFunction to make `ramp rate` plots.\n\n## 🛠️ infrastructure\n\n`.github/workflows` contains some CI actions.\n\n1. `docker-pipeline.yml`: Creates and publishes a docker image.\n\nWith any push to `main`, in order to deploy changes, the `Terraform Cloud` variable is updated with the commit reference and deployed to `AWS Elastic Beanstalk`.\n\n## Environmental Variables\n\n- DB_URL: The database url which will be queried for forecasts\n- password: The password for accessing the code\n- SHOW_PVNET_GSP_SUM: Option to show `pvnet_gsp_sum` model or not. This defaults to zero\n- REGION: Option can be UK or India. This effects the default values on the NWP and Satellite pages\n- ENVIRONMENT: Option can be `development` or `production`.\n  This effects the default values on the NWP and Satellite pages\n\n## Develop\n\nCurrently this repository is only used by OCF for internal metric calculations, as it contiains client information. We hope to make it more freely useable in the near future.\n\n### Tests\n\nTo run the tests, make sure you have `pytest` installed\n\n```bash\npip install pytest\n```\n\nand then you can run\n\n```bash\npytest\n```\n\n## Contributors and community\n\n[![issues badge](https://img.shields.io/github/issues/openclimatefix/elexonpy?color=FFAC5F)](https://github.com/openclimatefix/elexonpy/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc)\n\n- PR's are welcome! See the [Organisation Profile](https://github.com/openclimatefix) for details on contributing\n- Find out about our other projects in the [OCF Meta Repo](https://github.com/openclimatefix/ocf-meta-repo)\n- Check out the [OCF blog](https://openclimatefix.org/blog) for updates\n- Follow OCF on [LinkedIn](https://uk.linkedin.com/company/open-climate-fix)\n- OCF templete: (https://github.com/openclimatefix/ocf-template?tab=readme-ov-file#contributing-and-community)\n\nThe following folks have contributed to this repo.\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/suleman1412\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/37236131?v=4?s=100\" width=\"100px;\" alt=\"Suleman Karigar\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSuleman Karigar\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=suleman1412\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/peterdudfield\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/34686298?v=4?s=100\" width=\"100px;\" alt=\"Peter Dudfield\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003ePeter Dudfield\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"#projectManagement-peterdudfield\" title=\"Project Management\"\u003e📆\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/devsjc\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/47188100?v=4?s=100\" width=\"100px;\" alt=\"devsjc\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003edevsjc\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=devsjc\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"http://racheltipton.dev\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/86949265?v=4?s=100\" width=\"100px;\" alt=\"rachel tipton\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003erachel tipton\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=rachel-labri-tipton\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/braddf\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/41056982?v=4?s=100\" width=\"100px;\" alt=\"braddf\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003ebraddf\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=braddf\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/dfulu\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/41546094?v=4?s=100\" width=\"100px;\" alt=\"James Fulton\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eJames Fulton\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=dfulu\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/ADIMANV\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/68527614?v=4?s=100\" width=\"100px;\" alt=\"Aditya Sawant\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eAditya Sawant\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=ADIMANV\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/MAYANK12SHARMA\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/145884197?v=4?s=100\" width=\"100px;\" alt=\"MAYANK SHARMA\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eMAYANK SHARMA\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/openclimatefix/analysis-dashboard/commits?author=MAYANK12SHARMA\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n\n_Part of the [Open Climate Fix](https://github.com/orgs/openclimatefix/people) community._\n\n[![OCF Logo](https://cdn.prod.website-files.com/62d92550f6774db58d441cca/6324a2038936ecda71599a8b_OCF_Logo_black_trans.png)](https://openclimatefix.org)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fanalysis-dashboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenclimatefix%2Fanalysis-dashboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenclimatefix%2Fanalysis-dashboard/lists"}