{"id":17977619,"url":"https://github.com/martinmoene/scrape-vattenfall","last_synced_at":"2026-05-09T14:34:11.772Z","repository":{"id":164459068,"uuid":"639356027","full_name":"martinmoene/scrape-vattenfall","owner":"martinmoene","description":"Scrape Vattenfall web page with daily electricity usage.","archived":false,"fork":false,"pushed_at":"2024-01-04T15:51:10.000Z","size":473,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-09T12:17:58.252Z","etag":null,"topics":["ada","command-line-tool","python","text-processing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/martinmoene.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-05-11T09:55:35.000Z","updated_at":"2023-06-16T15:14:46.000Z","dependencies_parsed_at":"2023-10-15T10:30:32.184Z","dependency_job_id":null,"html_url":"https://github.com/martinmoene/scrape-vattenfall","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinmoene%2Fscrape-vattenfall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinmoene%2Fscrape-vattenfall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinmoene%2Fscrape-vattenfall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/martinmoene%2Fscrape-vattenfall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/martinmoene","download_url":"https://codeload.github.com/martinmoene/scrape-vattenfall/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247098945,"owners_count":20883287,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ada","command-line-tool","python","text-processing"],"created_at":"2024-10-29T17:28:41.609Z","updated_at":"2026-05-09T14:34:11.731Z","avatar_url":"https://github.com/martinmoene.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ScrapeVattenfall\n\nScrape Vattenfall web page with daily electricity usage.\n\n## Information on web page\n\nOne of the web pages on electricity usage available from the Vattenfall web site provides textual information on electricity usage and related monetary amounts for each day in a month. \n\nThe page is [Overzicht \u0026rarr; Verbruik](https://www.vattenfall.nl/service/mijn-vattenfall/#/vf/verbruik), select *Stroom*, *Dag*, month of interest, textual presentation (bullet list icon). The number of lines per day when no electricity is delivered to the network is lower than when it is ('Teruglevering'). See e.g. the entries for *4 mei* and *5 mei*.\n\n![Part of Vattenfall web page with electricity usage for each day in a month](media/202305-Verbruik%20details-Vattenfall-w600-circle.png)\n\nThe page presentation is dynamically generated, saving it as HTML doesn't provide one with the desired information. Hence I simply copy-paste the whole page in a text file, which I then convert using the [script](script/scrape_electricity_per_day_vattenfall.py) to the [CSV representation](#the-csv-presentation) described further below.\n\nSaved as text, this looks as follows, starting at the desired information, currently on line 27. The number of lines per day is 7 when no electricity is delivered to the network and it is 14 when it is ('Teruglevering').\n\n\u003cdetails\u003e\n\u003cdiv style=\"text-size=.6em\"\u003e\n\n```\nMei 2023\nDe variabele leveringskosten hieronder zijn berekend zonder het prijsplafond. Wat het prijsplafond voor u betekent, ziet u terug in uw persoonlijke termijnbedrag advies.\n\nLees meer\n1 mei € 2,37\nVariabele kosten\n7,162 kWh\n€ 2,86\nVaste kosten\n1 dg\n€ -0,49\n2 mei € 2,63\nVariabele kosten\n7,837 kWh\n€ 3,12\nVaste kosten\n1 dg\n€ -0,49\n3 mei € 2,10\nVariabele kosten\n6,505 kWh\n€ 2,59\nVaste kosten\n1 dg\n€ -0,49\n4 mei € 2,80\nVariabele kosten\n8,246 kWh\n€ 3,29\nVaste kosten\n1 dg\n€ -0,49\n5 mei € 0,16\nStroom\n6,359 kWh\n-\nTeruglevering\n-4,736 kWh\n-\nNetto verbruik\nVariabele kosten\n1,623 kWh\n€ 0,65\nVaste kosten\n1 dg\n€ -0,49\n6 mei € -1,62\nStroom\n4,564 kWh\n-\nTeruglevering\n-11,276 kWh\n-\nNetto verbruik\nVariabele kosten\n-6,712 kWh\n€ -1,13\nVaste kosten\n1 dg\n€ -0,49\n7 mei € -1,70\nStroom\n```\n\n\u003c/div\u003e\n\u003c/details\u003e\n\nThe points of interest are then as follows, with the start of a day at index 0.\n### A. Without Teruglevering\n\n| Index |  Line contents   | {Datum}                |\n| ----: | :--------------: | ---------------------- |\n|     0 | 1 januari € 2,75 | [date] {Totale kosten} |\n|     1 | Variabele kosten |                        |\n|     2 |    8,838 kWh     | {Levering}             |\n|     3 |      € 3,24      | {Variable Kosten}      |\n|     4 |   Vaste kosten   |                        |\n|     5 |       1 dg       |                        |\n|     6 |     € -0,49      | {Vaste kosten}         |\n\n### B. With Teruglevering\n\n| Index |  Line contents   | {Datum}                |\n| ----: | :--------------: | ---------------------- |\n|     0 |  8 mei € -2,15   | [date] {Totale kosten} |\n|     1 |      Stroom      |                        |\n|     2 |    3,985 kWh     | {Levering}             |\n|     3 |        -         |                        |\n|     4 |  Teruglevering   |                        |\n|     5 |   -13,862 kWh    | {Teruglevering}        |\n|     6 |        -         |                        |\n|     7 |  Netto verbruik  |                        |\n|     8 | Variabele kosten |                        |\n|     9 |    -9,877 kWh    | {Netto Verbruik}       |\n|    10 |     € -1,66      | {Variabele kosten}     |\n|    11 |   Vaste kosten   |                        |\n|    12 |       1 dg       |                        |\n|    13 |     € -0,49      | {Vaste kosten}         |\n\n\n## The CSV presentation\n\n```Text\nDatum;Levering [Wh];Teruglevering [Wh];Netto Verbruik [Wh];Vaste Kosten;Variable Kosten;Totale kosten\n01-01-23;8,838;0;8,838;-0.49;3.24;2.75\n...\n08-05-23;3,985;-13,862;-9,877;-0.49;-1,66;-2,15\n```\n\n## The Python script\n\nCurrently the [script](script/scrape_electricity_per_day_vattenfall.py) assumes the desired information starts on `line_month` (27). If needed, the scraping process can be made a bit more resilient. For now, let's hope the presentation does not often change.\n\nThe script's help screen:\n\n```Console\nusage: scrape_electricity_per_day_vattenfall.py [-h] [-v] [--csv-folder csv] [--output output] paths [paths ...]\n\nScrape given text file(s) with Vattenfall daily electricity usage and create file(s) in csv format. Single file output is to stdout default and can\nbe directed to a file using option '--output'. When multiple files are specified, output is to a file of the same name with the extension replaced\nwith '.csv'. Multiple file output can be directed to a folder using option '--csv-folder'.\n\npositional arguments:\n  paths             file(s) with copy-pasted web page text (file, folder, wildcard)\n\noptional arguments:\n  -h, --help        show this help message and exit\n  -v, --verbose     report file being processed (level 1), count (2), progress (3) (default: 0)\n  --csv-folder csv  folder to write csv files to (default: None)\n  --output output   output file in csv format (default: None)\n```\n\n## Ada version of the Python script (WIP)\n\nAs an excercise to learn Ada, I am writing an [Ada version](ada) of above Python script.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinmoene%2Fscrape-vattenfall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmartinmoene%2Fscrape-vattenfall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmartinmoene%2Fscrape-vattenfall/lists"}