{"id":22143431,"url":"https://github.com/cre-dev/pub-data-visualization","last_synced_at":"2026-03-07T11:02:35.627Z","repository":{"id":209453445,"uuid":"288209070","full_name":"cre-dev/pub-data-visualization","owner":"cre-dev","description":"The objective of this repository is to share with an MIT license the visualization tools used with public data and developed by the Wholesale Markets Surveillance Directorate (DSMG) of the Regulatory Commission of Energy (CRE). It can be used by final users such as developers and energy analysts.","archived":false,"fork":false,"pushed_at":"2022-09-28T13:51:35.000Z","size":5249,"stargazers_count":19,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-10-20T09:41:08.616Z","etag":null,"topics":["cre","electricity","energy","open-source","public-data","visualization"],"latest_commit_sha":null,"homepage":"https://www.cre.fr/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cre-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-08-17T14:56:41.000Z","updated_at":"2024-10-29T17:27:30.000Z","dependencies_parsed_at":"2023-11-27T13:13:49.886Z","dependency_job_id":null,"html_url":"https://github.com/cre-dev/pub-data-visualization","commit_stats":null,"previous_names":["cre-dev/pub-data-visualization"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cre-dev/pub-data-visualization","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cre-dev%2Fpub-data-visualization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cre-dev%2Fpub-data-visualization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cre-dev%2Fpub-data-visualization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cre-dev%2Fpub-data-visualization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cre-dev","download_url":"https://codeload.github.com/cre-dev/pub-data-visualization/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cre-dev%2Fpub-data-visualization/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30212103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T09:02:10.694Z","status":"ssl_error","status_checked_at":"2026-03-07T09:02:08.429Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cre","electricity","energy","open-source","public-data","visualization"],"created_at":"2024-12-01T22:10:33.042Z","updated_at":"2026-03-07T11:02:35.611Z","avatar_url":"https://github.com/cre-dev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Presentation\nThe objective of this repository is to share with an [MIT license](https://spdx.org/licenses/MIT.html) the visualization tools used with public data and developed by the Wholesale Markets Surveillance Directorate ([DSMG](https://www.cre.fr/en/CRE/organization)) of the Regulatory Commission of Energy ([CRE](https://www.cre.fr/en)). It can be used by final users such as developers and energy analysts.\n\nTo obtain the proposed interactive visualizations, the user should :\n* install the package, \n* download the input data,\n* run one of the scripts with ipython and her own choice of parameters.\n\nAll suggestions are welcome at [opensource\\[at\\]cre.fr](mailto:opensource@cre.fr).\n\n# Installation\nIt can be installed with :\n```\ncd ~/Downloads\ngit clone https://github.com/cre-dev/pub-data-visualization.git\ncd pub-data-visualization\npython3 -m venv venv\nsource venv/bin/activate\npython3 -m pip install --upgrade pip\npython3 -m pip install -e .\n```\nThe installation can then be tested with one of the following :\n```\npython3 scripts/weather/main_curve.py\n```\nor \n```\npython3 scripts/load/main_forecasting_error.py\n```\nThe scripts should terminate without any error.\nThey will create 3 folders in the home directory :\n```\n~/\n├── _energy_plots/       # for the plots\n├── _energy_public_data/ # for the raw data\n├── _energy_tmp_data/    # for the transformed data\n```\n\nHowever, ipython should then be preferred for interactive plots.\n\n# Download the input data\nThe data used for the visualizations proposed in this repository come from different public data sources.\nonly the data from eCO2mix and Météo-France are downloaded automatically.\nData from RTE and ENTSO-E should be downloaded manually.\n\n## eCO2mix\nData about the supply and demand equilibrium and provided by Réseau de Transport d’Electricité ([RTE](https://www.rte-france.com/eco2mix/telecharger-les-indicateurs)) through [eCO2mix](https://www.rte-france.com/eco2mix/telecharger-les-indicateurs) allow to illustrate the production and the consumption on the French electricity network.\nThey **can be downloaded automatically**.\nNo account is necessary.\n\n## ENTSO-E\nThe European Network of Transmission System Operators for Electricity ([ENTSO-E](https://www.entsoe.eu/)) publishes fundamental data on its [transparency platform](https://transparency.entsoe.eu/).\nThe source files used for the visualizations in this repository currently **have to be downloaded manually** with the [SFTP share](\u003chttps://transparency.entsoe.eu/content/static_content/Static content/knowledge base/SFTP-Transparency_Docs.html\u003e).\nAn account is necessary.\n\n## Météo-France\nAs the French national meteorological service, [Météo-France](http://meteofrance.com/) provides [observation data](https://donneespubliques.meteofrance.fr/?fond=produit\u0026id_produit=90\u0026id_rubrique=32) extracted from the Global Telecommunication System ([GTS](https://public.wmo.int/en/programmes/global-telecommunication-system)) of the World Meteorological Organization ([WMO](https://public.wmo.int/en)).\nThe data **can be downloaded automatically**.\nNo account is necessary.\n\n## RTE\nRTE publishes fundamental data about the French electricity transmission system.\nThe files currently **have to be downloaded manually** on the platform [RTE services portal](https://services-rte.com/en/download-data-published-by-rte.html).\nAn account is necessary.\n\n## Local organization of the data\nThe data have to be stored as follows :\n\n```\n~/_energy_public_data/\n├── 11_ENTSOE/\n│  ├── ActualGenerationOutputPerGenerationUnit_16.1.A/\n│  │  ├── 2021_01_ActualGenerationOutputPerGenerationUnit_16.1.A.csv\n│  │  ├── …\t \n│  ├── ActualTotalLoad_6.1.A/\n│  │  ├── 2021_01_ActualTotalLoad_6.1.A.csv\n│  │  ├── …\t \n│  ├── DayAheadPrices_12.1.D/\t \n│  │  ├── 2021_01_DayAheadPrices_12.1.D.csv\n│  │  ├── …\t \n│  ├── Outages/\t \t \n│  │  ├── UnavailabilityOfGenerationUnits_15.1.A_B/\t \n│  │  │  ├── 2021_01_UnavailabilityOfGenerationUnits_15.1.A_B.csv\n│  │  │  ├── …\t \n│  │  ├── UnavailabilityOfProductionUnits_15.1.C_D/\t \n│  │  │  ├── 2021_01_UnavailabilityOfProductionUnits_15.1.C_D.csv\n│  │  │  ├── …\t \n├──20_MeteoFrance/\t \t \n│  ├── synop/\t \t \n│  │  ├── postesSynop.csv\n│  │  ├── synop.201001.csv\n│  │  ├── …\t \n├──24_RTE/\t \t \t \n│  ├── Centrales_production_reference/\n│  │  ├── Centrales_production_reference.xls\n│  ├── DonneesIndisponibilitesProduction/\n│  │  ├── DonneesIndisponibilitesProduction_2010.xls\n│  │  ├── …\t \n│  ├── eCO2mix_RTE/\t \n│  │  ├── eCO2mix_RTE_Annuel-Definitif_2012.xls\n│  │  ├── …\t \n│  ├── ProductionGroupe/\t \n│  │  ├── ProductionGroupe_2012/\n│  │  │  ├── ProductionGroupe_2012-semestre1.xls\n│  │  │  ├── …\n│  │  ├── …\n```\n\n\n# How-to : ready-to-run examples\nIn this repository, we propose a set of modules that read, format, transform and plot the input data from different public sources.\nWe also provide ready-to-run visualization scripts as illustrated below.\nThe parameters therein can be modified by the user.\n\n## Indices\n\n### Day-ahead fixing prices\n![Auction prices](examples/indices/prices.png)\nThis figure, that represents the fixing prices of the day-ahead auctions, is obtained by running `scripts/indices/main_price.py`.\nThe data, provided by ENTSO-E, currently have to be downloaded manually.\n\n## Load\n\n### Load curve\n![Load curve](examples/load/power.png)\nThis figure, obtained by running `scripts/load/main_power.py`, is a mere representation of the load curve.\nData from eCO2mix and ENTSO-E can serve as inputs.\n\n### Day-ahead forecasting error\n![Day-ahead forecasting error](examples/load/forecasting_error.png)\nThis figure represents the national load forecasting error and is obtained by running `scripts/load/main_forecasting_error.py`.\nThe data are provided by eCO2mix.\n\n\n## Outages\n\n### Animated view of the unavailability\n![Animated view of the unavailability](examples/outages/animated_availability.png)\nThis figure is obtained by running `scripts/outages/main_animated_availability.py`.\nThe data have to be downloaded manually from ENTSO-E or RTE platforms.\n\n### Evolution of the mean unavailability\n![Evolution of the mean unavailability](examples/outages/evolution_mean_availability.png)\nThis figure is obtained by running `scripts/outages/main_evolution_mean_availability.py`.\nThe data have to be downloaded manually from ENTSO-E or RTE platforms.\n\n### Expected availability program of a given unit\n![Expected availability program of a given unit](examples/outages/expected_program.png)\nThis figure is obtained by running `scripts/outages/main_expected_program.py`.\nThe data have to be downloaded manually from ENTSO-E or RTE platforms.\n\n### Availability programs\n![Availability programs](examples/outages/incremental_programs.png)\nThis figure is obtained by running `scripts/outages/main_incremental_programs.py`.\nThe data have to be downloaded manually from ENTSO-E or RTE platforms.\n\n### Regression delays\n![Regression delays](examples/outages/regression_delays.png)\nThis figure is obtained by running `scripts/outages/main_regression_delays.py`.\nThe data have to be downloaded manually from ENTSO-E or RTE platforms.\nIt displays a linear fit between the initially announced and the finally observed length of the outages.\nThe coefficient being obtained with the minimization of a squared error, outliers have a significant effect.\n\n\n## Production data\n\n### Unit production\n![Unit production](examples/production/power.png)\nThis figure is obtained by running `scripts/production/main_power.py`.\nIt can be used with data provided by eCO2mix, ENTSO-E or RTE.\n\n\n## Weather data\n\n### National mean weather\n![Weather curve](examples/weather/curve.png)\nThis figure is obtained by running `scripts/weather/main_curve.py`.\nThe data, provided by Météo-France, are downloaded automatically.\n\n### Distribution of the temperature\n![Distribution temperature](examples/weather/distribution_temperature.png)\nThis figure is obtained by running `scripts/weather/main_distribution.py`.\nThe data, provided by Météo-France, are downloaded automatically.\n\n\n## Multiplots\n\n### Spot report\n![Spot report](examples/multiplots/spot_report.png)\nThis figure is obtained by running `scripts/multiplots/main_spot_report.py`.\nAs it mixes data from different sources, the data from ENTSO-E and RTE have to be downloaded manually.\n\n### Announced availability and observed production of a given unit\n![Transparent production](examples/multiplots/transparent_production.png)\nThis figure is obtained by running `scripts/multiplots/main_transparent_production.py`.\nThe data from RTE have to be downloaded manually.\n\n### 2D-distribution of price and load\n![Price and Load](examples/multiplots/scatter_price_load.png)\nThis figure is obtained by running `scripts/multiplots/main_scatter_price_load.py`.\nThe data from ENTSO-E and RTE have to be downloaded manually.\n\n### 2D-distribution of price and production\n![Price and Production](examples/multiplots/scatter_price_production.png)\nThis figure is obtained by running `scripts/multiplots/main_scatter_price_production.py`.\nThe data from ENTSO-E and RTE have to be downloaded manually.\n\n### 2D-distribution of price and weather\n![Price and weather](examples/multiplots/scatter_price_weather.png)\nThis figure is obtained by running `scripts/multiplots/main_scatter_price_weather.py`.\nThe data from ENTSO-E have to be downloaded manually.\n\n\n\n\n# Description of the code\n\n## Documentation\nAn html documentation generated with Sphinx can be found in ./doc/pub_data_visualization.html.\n\n## Parameters of the scripts\n\n| Variable name                  | Type                            | Possible values                                                 | Purpose                                       |\n| ---                            | ---                             | ---                                                             | ---                                           |\n| close                          | bool                            | True; False\t                                             | Close the figure after saving                 |\n| contract_delivery_begin_year   | int                             | 2018; …                                                         | -                                             |\n| contract_delivery_period_index | int\t                           | depends on the selected contract_product                        | -                                             |\n| contract_frequency             | string\t                   | \"M\"; \"Q\"; …\t                                             | -                                             |\n| contract_profile               | string\t                   | \"BASE\"; \"PEAK\" …                                                | -                                             |\n| data_source_auctions           | string\t                   | \"ENTSOE\"                                                        | -                                             |\n| data_source_load               | string\t                   | \"eCO2mix\"; \"ENTSOE\"                                             | -                                             |\n| data_source_outages            | string                          | \"ENTSOE\"; \"RTE\"                                                 | -                                             |\n| data_source_production         | string                          | \"eCO2mix\"; \"ENTSOE\"; \"RTE\"                                      | -                                             |\n| data_source_weather            | string                          | \"MétéoFrance\"                                                   | -                                             |\n| delivery_begin_dt_max          | None or localized pd.Timestamp  | -                                                               | -                                             |\n| delivery_end_dt_min            | None or localized pd.Timestamp  | -                                                               | -                                             |\n| diff_init                      | bool                            | True; False                                                     | Plot the differences between pairs of dates   |\n| figsize                        | (int,int)                       | (8,6)                                                           | Figure size                                   |\n| folder_out                     | path                            | global_var.path_plots                                           | Plots output                                  |\n| load_nature                    | string                          | \"load forecast D-1 (GW)\";\"load forecast D-0 (GW)\"; \"load (GW)\"  | -                                             |\n| map_code                       | string or list of strings       | \"FR\"; … or [\"FR\", \"GB\", \"BE\"…]                                  | -                                             |\n| map_code_auctions              | string or list of strings       | \"FR\"; … or [\"FR\", \"GB\", \"BE\"…]                                  | -                                             |\n| producer_outages               | None or string                  | any producer that publishes                                     | -                                             |\n| production_dt_max              | None or localized pd.Timestamp  | -                                                               | -                                             |\n| production_dt_min              | None or localized pd.Timestamp  | -                                                               | -                                             |\n| production_nature              | string                          | \"production (GW)\"                                               | -                                             |\n| production_source              | None or string                  | \"biomass\"; \"solar\"; …                                           | -                                             |\n| publication_dt_max             | None or localized pd.Timestamp  | -                                                               | -                                             |\n| publication_dt_min             | None or localized pd.Timestamp  | -                                                               | -                                             |\n| smoother                       | string or pd.Timedelta          | \"basic\";                                                        | For aesthetic purposes                        |\n| unit_name                      | None or string                  | any production unit                                             | -                                             |\n| viewpoint_dt_extrapolate       | list of localized pd.Timestamp  | -                                                               | Position plotted as seen from these dates     |\n| weather_nature                 | string                          | \"observation\"                                                   | -                                             |\n| weather_quantity               | string                          | \"nebulosity (%)\"; '\"temperature (°C)\"; \"wind_speed (m/s)\"       | -                                             |\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcre-dev%2Fpub-data-visualization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcre-dev%2Fpub-data-visualization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcre-dev%2Fpub-data-visualization/lists"}