https://github.com/openclimatefix/analysis-dashboard
This is a Streamlit app for the OCF team that reports database statistics
https://github.com/openclimatefix/analysis-dashboard
Last synced: 12 months ago
JSON representation
This is a Streamlit app for the OCF team that reports database statistics
- Host: GitHub
- URL: https://github.com/openclimatefix/analysis-dashboard
- Owner: openclimatefix
- License: mit
- Created: 2023-02-14T12:14:31.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-06-13T12:57:15.000Z (12 months ago)
- Last Synced: 2025-06-13T13:52:12.439Z (12 months ago)
- Language: Python
- Size: 3.13 MB
- Stars: 5
- Watchers: 3
- Forks: 12
- Open Issues: 21
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# uk analysis dashboard
[](#contributors-)
[](https://github.com/openclimatefix#how-easy-is-it-to-get-involved)
Internal dashboard for OCF to track forecast statistics and log the historical data of the forecast performance.
The analysis dashboard is a tool that was developed for OCF’s internal use and continues to evolve.
Built with [Streamlit](https://streamlit.io/), a Python-based framework made specifically for creating data apps, the dashboard tracks and displays Quartz Solar and other data model statistics, such as mean absolute error (MAE), normalized mean absolute error (nMAE) for all the client sites. The database provides the error statistic using comparing the live generation with the forecast provided. Internally it has the option of chosing the forecast horizion to check the performance with genration. The larger the error, the less accurate the forecast.
Thanks to the analysis dashboard, OCF has a valuable feedback tool for understanding the accuracy of the forecasts being provided to it's clients.
## Installation
### **Manual Installation**
You can install the `analysis-dashboard` package directly from GitHub.
In the main project folder, install requirements:
```shell
pip install -r requirements.txt
```
Run `streamlit hello` to check that Streamlit installed. A "Welcome to Streamlit!" page should open in the browser.
Create a login secret: `
```shell
echo "password = example" > src/.streamlit/secrets.toml
```
## Database connection
To run the app locally, you'll need to connect it to the `forecast development database`
OCF team members can connect to the `forecast development database` using [these Notion instructions](https://www.notion.so/openclimatefix/Connecting-to-AWS-RDS-bf35b3fbd61f40df9c974c240e042354). Add `DB_URL= (db_url from notion documents)` to a `secrets.toml` file. Follow the instructions in the Notion document to connect to the database v.
Run app:
```shell
cd src && streamlit run main.py
```
---
### **Using Docker Compose**
This method uses Docker Compose to set up the app and its environment automatically.
**Prerequisites:**
You need to have Docker and Docker Compose installed on your machine. If you don't have them, you can download them from the [Docker website](https://www.docker.com/get-started).
**Steps:**
1. Clone the repository and navigate to the project folder:
```shell
git clone https://github.com/openclimatefix/analysis-dashboard.git
cd analysis-dashboard
```
2. Create a `.env` file in the root directory and add the following environment variables:
```shell
# DB_URL=your-database-url # Optional, if not available, you can skip this line
REGION=india # Choose 'india' or 'uk'
ENVIRONMENT=development # or 'production'
password=example # Set your password here
SHOW_PVNET_GSP_SUM=0 # Set this to 1 if you want to show pvnet_gsp_sum model
```
3. Create a `secrets.toml` file in the `src/.streamlit` directory and add the following line:
```shell
echo "password = example" > src/.streamlit/secrets.toml
```
4. Build the Docker image and start the app:
```shell
docker-compose up --build
```
5. Open your browser and go to `http://localhost:8501` to view the app.
6. To stop the app, press `Ctrl+C` in the terminal, and then run:
```shell
docker-compose down
```
## Files
### main.py
`main.py` contains functions for the `home page` of the app, which focuses on MAE for the OCF `Quartz Solar` forecast.
### main_india.py
`main_india.py` contains functions for the `home page` of the app, which focuses on MAE for the OCF `Quartz Energy` forecast.
### forecast.py
`forecast.py` contains functions for the `forecast page`. The forecast page looks at how well each of OCF's forecast models is performing compared to `PVLive updated` truth values.
### status.py
`status.py` contains functionality for the `status pagwe` and allows the OCF team to update the forecast status in the database. This is one of the advantages of using an interface like Streamlit, facilitating status updates in a database.
### auth.py
`auth.py` contains code for the basic authenticaion that's been put in place.
### pvsite_forecast.py
`pvsite_forecast.py` contains the formulas and the metrics used to calculate MAE, nMAE and penalty incured against all sites.
### site_toolbox.py
`site_toolbox.py` is a page on the dashboard that can be used to get details of any particular site that OCF provides forecast to.
### plots/make_pinball_and_exceedance_plots.py
Function to make `pinball` and exceedance plots. This shows how good the probabilistic forecasts are doing.
### plots/ramp_rate.py
Function to make `ramp rate` plots.
## 🛠️ infrastructure
`.github/workflows` contains some CI actions.
1. `docker-pipeline.yml`: Creates and publishes a docker image.
With any push to `main`, in order to deploy changes, the `Terraform Cloud` variable is updated with the commit reference and deployed to `AWS Elastic Beanstalk`.
## Environmental Variables
- DB_URL: The database url which will be queried for forecasts
- password: The password for accessing the code
- SHOW_PVNET_GSP_SUM: Option to show `pvnet_gsp_sum` model or not. This defaults to zero
- REGION: Option can be UK or India. This effects the default values on the NWP and Satellite pages
- ENVIRONMENT: Option can be `development` or `production`.
This effects the default values on the NWP and Satellite pages
## Develop
Currently this repository is only used by OCF for internal metric calculations, as it contiains client information. We hope to make it more freely useable in the near future.
### Tests
To run the tests, make sure you have `pytest` installed
```bash
pip install pytest
```
and then you can run
```bash
pytest
```
## Contributors and community
[](https://github.com/openclimatefix/elexonpy/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc)
- PR's are welcome! See the [Organisation Profile](https://github.com/openclimatefix) for details on contributing
- Find out about our other projects in the [OCF Meta Repo](https://github.com/openclimatefix/ocf-meta-repo)
- Check out the [OCF blog](https://openclimatefix.org/blog) for updates
- Follow OCF on [LinkedIn](https://uk.linkedin.com/company/open-climate-fix)
- OCF templete: (https://github.com/openclimatefix/ocf-template?tab=readme-ov-file#contributing-and-community)
The following folks have contributed to this repo.
Suleman Karigar
💻
Peter Dudfield
📆
devsjc
💻
rachel tipton
💻
braddf
💻
James Fulton
💻
Aditya Sawant
💻
MAYANK SHARMA
📖
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
_Part of the [Open Climate Fix](https://github.com/orgs/openclimatefix/people) community._
[](https://openclimatefix.org)