{"id":13722349,"url":"https://github.com/OSUKED/ElexonDataPortal","last_synced_at":"2025-05-07T15:30:40.465Z","repository":{"id":39801261,"uuid":"189842391","full_name":"OSUKED/ElexonDataPortal","owner":"OSUKED","description":"Python wrapper for the Elexon/BMRS API","archived":false,"fork":false,"pushed_at":"2023-09-26T17:58:03.000Z","size":7238144,"stargazers_count":56,"open_issues_count":11,"forks_count":15,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-28T13:05:07.440Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://osuked.github.io/ElexonDataPortal","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OSUKED.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-06-02T12:14:48.000Z","updated_at":"2025-04-10T14:01:58.000Z","dependencies_parsed_at":"2024-01-08T01:07:47.199Z","dependency_job_id":null,"html_url":"https://github.com/OSUKED/ElexonDataPortal","commit_stats":{"total_commits":23304,"total_committers":3,"mean_commits":7768.0,"dds":0.000514933058702316,"last_synced_commit":"d8be59c56dda41bd8f2a9efbb3620b4b563710da"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSUKED%2FElexonDataPortal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSUKED%2FElexonDataPortal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSUKED%2FElexonDataPortal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OSUKED%2FElexonDataPortal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OSUKED","download_url":"https://codeload.github.com/OSUKED/ElexonDataPortal/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252210989,"owners_count":21712308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T01:01:27.675Z","updated_at":"2025-05-07T15:30:35.447Z","avatar_url":"https://github.com/OSUKED.png","language":"Jupyter Notebook","funding_links":[],"categories":["Energy Systems"],"sub_categories":["Energy Data Accessibility and Integration"],"readme":"# Elexon Data Portal\n\n[![DOI](https://zenodo.org/badge/189842391.svg)](https://zenodo.org/badge/latestdoi/189842391) [![Binder](https://notebooks.gesis.org/binder/badge_logo.svg)](https://notebooks.gesis.org/binder/v2/gh/OSUKED/ElexonDataPortal/master?urlpath=lab%2Ftree%2Fnbs%2F08-quick-start.ipynb) [![PyPI version](https://badge.fury.io/py/ElexonDataPortal.svg)](https://badge.fury.io/py/ElexonDataPortal)\n\nThe `ElexonDataPortal` library is a Python Client for retrieving data from the Elexon/BMRS API. The library significantly reduces the complexity of interfacing with the Elexon/BMRS API through the standardisation of parameter names and orchestration of multiple queries when making requests over a date range. To use the `ElexonDataPortal` you will have to register for an Elexon API key which can be done [here](https://www.elexonportal.co.uk/registration/newuser). \n\n\u003cbr\u003e\n\u003cbr\u003e\n\n### Installation\n\nThe library can be easily installed from PyPi, this can be done using:\n\n```bash\npip install ElexonDataPortal\n```\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n### Getting Started\n\nWe'll begin by initialising the API `Client`. The key parameter to pass here is the `api_key`, alternatively this can be set by specifying the environment variable `BMRS_API_KEY` which will then be loaded automatically.\n\n```python\nfrom ElexonDataPortal import api\n\nclient = api.Client('your_api_key_here')\n```\n\n\u003cbr\u003e\n\nNow that the client has been initialised we can make a request! \n\nOne of the key abstractions within the `ElexonDataPortal` library is the handling of multiple requests over a date range specified through the `start_date` and `end_date` parameters. Each response will be automatically cleaned and parsed, then concatenated into a single Pandas DataFrame. If a settlement period and date column can be identified in the returned data then a new column will be added with the local datetime for each data-point. N.b. that if passed as a string the start and end datetimes will be assumed to be in the local timezone for the UK\n\n```python\nstart_date = '2020-01-01'\nend_date = '2020-01-01 1:30'\n\ndf_B1610 = client.get_B1610(start_date, end_date)\n\ndf_B1610.head(3)\n```\n\n|    | documentType      | businessType   | processType   | timeSeriesID          | curveType                   | settlementDate   | powerSystemResourceType   | registeredResourceEICCode   | marketGenerationUnitEICCode   | marketGenerationBMUId   | marketGenerationNGCBMUId   | bMUnitID    | nGCBMUnitID   | activeFlag   | documentID              |   documentRevNum | resolution   | start      | end        |   settlementPeriod |   quantity | local_datetime            |\n|---:|:------------------|:---------------|:--------------|:----------------------|:----------------------------|:-----------------|:--------------------------|:----------------------------|:------------------------------|:------------------------|:---------------------------|:------------|:--------------|:-------------|:------------------------|-----------------:|:-------------|:-----------|:-----------|-------------------:|-----------:|:--------------------------|\n|  0 | Actual generation | Production     | Realised      | ELX-EMFIP-AGOG-TS-212 | Sequential fixed size block | 2020-01-01       | Generation                | 48W000CAS-BEU01F            | 48W000CAS-BEU01F              | M_CAS-BEU01             | CAS-BEU01                  | M_CAS-BEU01 | CAS-BEU01     | Y            | ELX-EMFIP-AGOG-22495386 |                1 | PT30M        | 2020-01-01 | 2020-01-01 |                  1 |     18.508 | 2020-01-01 00:00:00+00:00 |\n|  1 | Actual generation | Production     | Realised      | ELX-EMFIP-AGOG-TS-355 | Sequential fixed size block | 2020-01-01       | Generation                | 48W00000STLGW-3A            | 48W00000STLGW-3A              | T_STLGW-3               | STLGW-3                    | T_STLGW-3   | STLGW-3       | Y            | ELX-EMFIP-AGOG-22495386 |                1 | PT30M        | 2020-01-01 | 2020-01-01 |                  1 |     28.218 | 2020-01-01 00:00:00+00:00 |\n|  2 | Actual generation | Production     | Realised      | ELX-EMFIP-AGOG-TS-278 | Sequential fixed size block | 2020-01-01       | Generation                | 48W00000GNFSW-1H            | 48W00000GNFSW-1H              | T_GNFSW-1               | GNFSW-1                    | T_GNFSW-1   | GNFSW-1       | Y            | ELX-EMFIP-AGOG-22495386 |                1 | PT30M        | 2020-01-01 | 2020-01-01 |                  1 |     29.44  | 2020-01-01 00:00:00+00:00 |\n\n\u003cbr\u003e\n\nIf you've previously written your own code for extracting data from the Elexon/BMRS API then you may be wondering where some of the normal parameters you pass have gone. The reduction in the parameters passed are due to 4 core drivers:\n\n* Standardisation of date range parameter names\n* Removal of the need to specify `ServiceType`\n* Automatic passing of `APIKey` after client initialisation\n* Shipped with sensible defaults for all remaining parameters\n\nThe full list of data streams that are able to be requested can be found [here](#data-stream-descriptions). If you wish to make requests using the raw methods these are available through the `ElexonDataportal.dev.raw` module.\n\nFurther information can be found in the [Quick Start guide](https://osuked.github.io/ElexonDataPortal/08-quick-start/).\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n### What's Changed in v2\n\nThe latest release of the library includes a full rewrite of the code-base. We have endeavoured to make the new API as intuitive as possible but that has required breaking changes from v1, if you wish to continue using the historic library use `pip install ElexonDataPortal==1.0.4`. N.b v1 will not be maintained going forward, you are advised to change over to v2.0.0+. \n\nThe key feature changes are:\n\n* Coverage of more BMRS streams \n* Automated default values\n* Cleaner client API\n* A larger range of request types are compatible with the date range orchestrator\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n### Programmatic Library Generation\n\nOne of the core features within the `ElexonDataPortal` library is that it is *self-generating*, by which we mean it can rebuild itself (including any new API request methods) from scratch using only the `endpoints.csv` spreadsheet. As well as generating the Python Client library a `BMRS_API.yaml` file is created, this provides an OpenAPI specification representation of the Elexon/BMRS API. In turn this allows us to automatically generate documentation, as well as run tests on the API itself to ensure that everything is working as expected - during this process we identified and corrected several small errors in the API documentation provided by Elexon. \n\nTo rebuild the library simply run the following in the root directory: \n\n```bash\npython -m ElexonDataPortal.rebuild\n```\n\n\u003cbr\u003e\n\nN.b. If you wish to develop the library further or use any of the programmatic library generation functionality then please install the development version of the library using:\n\n```bash\npip install ElexonDataPortal[dev]\n```\n\nIf you are not installing into a fresh environment it is recommended you install `pyyaml` and `geopandas` using conda to avoid any dependency conflicts. In future we are looking to release `ElexonDataPortal` as a conda package to avoid these issues.\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n### Data Stream Descriptions\n\nThe following table describes the data streams that are currently retreivable through the API. The client method to retrieve data from a given stream follows the naming convention `get_{stream-name}`.\n\n| Stream                 | Description                                                    |\n|:-----------------------|:---------------------------------------------------------------|\n| B0610                  | Actual Total Load per Bidding Zone                             |\n| B0620                  | Day-Ahead Total Load Forecast per Bidding Zone                 |\n| B0630                  | Week-Ahead Total Load Forecast per Bidding Zone                |\n| B0640                  | Month-Ahead Total Load Forecast Per Bidding Zone               |\n| B0650                  | Year Ahead Total Load Forecast per Bidding Zone                |\n| B0710                  | Planned Unavailability of Consumption Units                    |\n| B0720                  | Changes In Actual Availability Of Consumption Units            |\n| B0810                  | Year Ahead Forecast Margin                                     |\n| B0910                  | Expansion and Dismantling Projects                             |\n| B1010                  | Planned Unavailability In The Transmission Grid                |\n| B1020                  | Changes In Actual Availability In The Transmission Grid        |\n| B1030                  | Changes In Actual Availability of Offshore Grid Infrastructure |\n| B1320                  | Congestion Management Measures Countertrading                  |\n| B1330                  | Congestion Management Measures Costs of Congestion Management  |\n| B1410                  | Installed Generation Capacity Aggregated                       |\n| B1420                  | Installed Generation Capacity per Unit                         |\n| B1430                  | Day-Ahead Aggregated Generation                                |\n| B1440                  | Generation forecasts for Wind and Solar                        |\n| B1510                  | Planned Unavailability of Generation Units                     |\n| B1520                  | Changes In Actual Availability of Generation Units             |\n| B1530                  | Planned Unavailability of Production Units                     |\n| B1540                  | Changes In Actual Availability of Production Units             |\n| B1610                  | Actual Generation Output per Generation Unit                   |\n| B1620                  | Actual Aggregated Generation per Type                          |\n| B1630                  | Actual Or Estimated Wind and Solar Power Generation            |\n| B1720                  | Amount Of Balancing Reserves Under Contract Service            |\n| B1730                  | Prices Of Procured Balancing Reserves Service                  |\n| B1740                  | Accepted Aggregated Offers                                     |\n| B1750                  | Activated Balancing Energy                                     |\n| B1760                  | Prices Of Activated Balancing Energy                           |\n| B1770                  | Imbalance Prices                                               |\n| B1780                  | Aggregated Imbalance Volumes                                   |\n| B1790                  | Financial Expenses and Income For Balancing                    |\n| B1810                  | Cross-Border Balancing Volumes of Exchanged Bids and Offers    |\n| B1820                  | Cross-Border Balancing Prices                                  |\n| B1830                  | Cross-border Balancing Energy Activated                        |\n| BOD                    | Bid Offer Level Data                                           |\n| CDN                    | Credit Default Notice Data                                     |\n| DERSYSDATA             | Derived System Data                                            |\n| DETSYSPRICES           | Detailed System Prices                                         |\n| DEVINDOD               | Daily Energy Volume Data                                       |\n| DISBSAD                | Balancing Services Adjustment Action Data                      |\n| FORDAYDEM              | Forecast Day and Day Ahead Demand Data                         |\n| FREQ                   | Rolling System Frequency                                       |\n| FUELHH                 | Half Hourly Outturn Generation by Fuel Type                    |\n| MELIMBALNGC            | Forecast Day and Day Ahead Margin and Imbalance Data           |\n| MID                    | Market Index Data                                              |\n| MessageDetailRetrieval | REMIT Flow - Message List Retrieval                            |\n| MessageListRetrieval   | REMIT Flow - Message List Retrieval                            |\n| NETBSAD                | Balancing Service Adjustment Data                              |\n| NONBM                  | Non BM STOR Instructed Volume Data                             |\n| PHYBMDATA              | Physical Data                                                  |\n| SYSDEM                 | System Demand                                                  |\n| SYSWARN                | System Warnings                                                |\n| TEMP                   | Temperature Data                                               |\n| WINDFORFUELHH          | Wind Generation Forecast and Out-turn Data                     |","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOSUKED%2FElexonDataPortal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOSUKED%2FElexonDataPortal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOSUKED%2FElexonDataPortal/lists"}