{"id":13586169,"url":"https://github.com/UCA-Datalab/nilm-thresholding","last_synced_at":"2025-04-07T14:33:46.356Z","repository":{"id":42392568,"uuid":"259956039","full_name":"UCA-Datalab/nilm-thresholding","owner":"UCA-Datalab","description":":warning: This repository is no longer actively maintained. It previously dealt with Non-Intrusive Load Monitoring (NILM), focusing on predicting household appliance status from aggregated power load data. We explored different thresholding methods and evaluated deep learning models for regression and classification tasks.","archived":false,"fork":false,"pushed_at":"2023-09-14T09:33:37.000Z","size":701,"stargazers_count":33,"open_issues_count":8,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-02-14T21:24:02.117Z","etag":null,"topics":["convolutional-neural-networks","deep-learning","domestic-appliances","gru","monitoring","neural-network","nilm","nilm-algorithms","pytorch","recurrent-neural-networks","tensorflow","thresholding","thresholding-methods"],"latest_commit_sha":null,"homepage":"https://doi.org/10.1007/s11227-023-05149-8","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UCA-Datalab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-29T14:54:25.000Z","updated_at":"2024-08-01T16:32:18.686Z","dependencies_parsed_at":"2024-08-01T16:42:19.397Z","dependency_job_id":null,"html_url":"https://github.com/UCA-Datalab/nilm-thresholding","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UCA-Datalab%2Fnilm-thresholding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UCA-Datalab%2Fnilm-thresholding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UCA-Datalab%2Fnilm-thresholding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UCA-Datalab%2Fnilm-thresholding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UCA-Datalab","download_url":"https://codeload.github.com/UCA-Datalab/nilm-thresholding/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247670502,"owners_count":20976587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","deep-learning","domestic-appliances","gru","monitoring","neural-network","nilm","nilm-algorithms","pytorch","recurrent-neural-networks","tensorflow","thresholding","thresholding-methods"],"created_at":"2024-08-01T15:05:21.992Z","updated_at":"2025-04-07T14:33:46.042Z","avatar_url":"https://github.com/UCA-Datalab.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003c!-- README template: https://github.com/othneildrew/Best-README-Template --\u003e\n\n\u003c!-- PROJECT SHIELDS --\u003e\n\u003c!--\n*** I'm using markdown \"reference style\" links for readability.\n*** Reference links are enclosed in brackets [ ] instead of parentheses ( ).\n*** See the bottom of this document for the declaration of the reference variables\n*** for contributors-url, forks-url, etc. This is an optional, concise syntax you may use.\n*** https://www.markdownguide.org/basic-syntax/#reference-style-links\n--\u003e\n![Project Status](https://img.shields.io/badge/Status-Not%20Maintained-red.svg)\n\n[![Contributors][contributors-shield]][contributors-url]\n[![Forks][forks-shield]][forks-url]\n[![Stargazers][stars-shield]][stars-url]\n[![Issues][issues-shield]][issues-url]\n[![LinkedIn][linkedin-shield]][linkedin-url]\n\n\u003c!-- PROJECT LOGO --\u003e\n\u003cbr /\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/UCA-Datalab\"\u003e\n    \u003cimg src=\"images/logo.png\" alt=\"Logo\" width=\"400\" height=\"80\"\u003e\n  \u003c/a\u003e\n\n  \u003ch3 align=\"center\"\u003eNILM: classification VS regression\u003c/h3\u003e\n\u003c/p\u003e\n\n:warning: **Notice:** This project is no longer actively maintained. Feel free to use the existing code, but please note that there won't be further updates or bug fixes.\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003cdetails open=\"open\"\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\n      \u003ca href=\"#about-the-project\"\u003eAbout The Project\u003c/a\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#getting-started\"\u003eGetting Started\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#create-the-environment\"\u003eCreate the Environment\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#datasets\"\u003eDatasets\u003c/a\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#uk-dale\"\u003eUK-DALE\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n      \u003cul\u003e\n        \u003cli\u003e\u003ca href=\"#pecan-street-dataport\"\u003ePecan Street Dataport\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n   \u003cli\u003e\u003ca href=\"#preprocess-the-data\"\u003ePreprocess the Data\u003c/a\u003e\u003c/li\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\n      \u003ca href=\"#train\"\u003eTrain\u003c/a\u003e\n      \u003cul\u003e\n         \u003cli\u003e\u003ca href=\"#reproduce-the-paper\"\u003eReproduce the Paper\u003c/a\u003e\u003c/li\u003e\n         \u003cli\u003e\u003ca href=\"#thresholding-methods\"\u003eThresholding Methods\u003c/a\u003e\u003c/li\u003e\n      \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#publications\"\u003ePublications\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#contact\"\u003eContact\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#acknowledgements\"\u003eAcknowledgements\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\n## About the project\n\nNon-Intrusive Load Monitoring (NILM)  aims to predict the status\nor consumption of  domestic appliances in a household only by knowing\nthe aggregated power load. NILM can be formulated as regression problem\nor most often as a classification problem. Most datasets gathered\nby smart meters allow to  define naturally a regression problem,\nbut the corresponding classification problem  is a derived one,\nsince it requires a conversion from the power signal to the status of each\ndevice by a thresholding method. We treat three different thresholding\nmethods to perform this task, discussing their differences on various\ndevices from the UK-DALE dataset. We analyze the performance of\ndeep learning state-of-the-art architectures on both the regression and\nclassification problems, introducing criteria to select the most convenient\nthresholding method.\n\n## Getting started\n### Create the Environment\n\nTo create the environment using Conda:\n\n  1. Install miniconda\n     \n     ```\n     curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | bash\n     ```\n\n     Say yes to everything and accept default locations. Refresh bash shell with `bash -l`\n\n  2. Update conda\n     \n      ```\n      conda update -n base -c defaults conda\n      ```\n\n  3. Clone this repository and cd into the folder\n\n  4. Create and activate conda environment (removing previously existing env of the same name)\n     \n       ```\n       conda remove --name nilm-thresholding --all\n       conda env create -f environment.yml --force\n       conda activate nilm-thresholding\n       ```\n \n## Datasets\n\n### UK-DALE\n\nUK-DALE dataset is hosted on the following link:\n[https://data.ukedc.rl.ac.uk/browse/edc/efficiency/residential\n/EnergyConsumption/Domestic/UK-DALE-2017/UK-DALE-FULL-disaggregated](https://data.ukedc.rl.ac.uk/browse/edc/efficiency/residential/EnergyConsumption/Domestic/UK-DALE-2017/UK-DALE-FULL-disaggregated)\n\nThe files needed by this module are *ukdale.zip* and *ukdale.h5.zip*.\nDownload both and unzip them in a folder named [data](/data) inside the root.\nOnce you are done, your local directory should look like this:\n\n```\nnilm-thresholding\n|_ nilmth\n   |_ [python scripts and subfolders]\n|_ data\n   |_ ukdale\n      |_ [house_1 to house_5]\n   |_ ukdale.h5\n```\n\nCredit: [Jack Kelly](https://jack-kelly.com/data/)\n\n### Pecan Street Dataport\n\nWe are aiming to include this dataset in a future release. You can check the issue here: [https://github.com/UCA-Datalab/nilm-thresholding/issues/8](https://github.com/UCA-Datalab/nilm-thresholding/issues/8)\n\nAny help and suggestions are welcome!\n\nCredit: [Pecan Street](https://dataport.pecanstreet.org/)\n\n## Preprocess the Data\n\nOnce downloaded the raw data from any of the sources above,\nyou must preprocess it.\nThis is done by running the following script:\n\n```\npython nilmth/preprocess.py\n```\n\nThis will generate a new folder, named 'data-prep',\ncontaining all the preprocessed data.\n\n## Train\n\nThe folder [nilmth](/nilmth) contains an executable script to train the\n models. Run the following line on the root folder\n(make sure to have the enviroment active and the data downloaded):\n\n```\npython nilmth/train.py\n```\n\nThis will train the CONV model using the default parameters.\nThe script stores several outputs in the [outputs folder](/outputs),\nincluding:\n- .txt files with the model scores over the validation data\n- .pth files containing the model weights\n- .png files with a section of the validation data and the model's prediction\n\nThe list with all the available parameters and their default values is stored in the\n [configuration file](nilmth/config.toml).\n\nIf you want to use your own set of parameters, duplicate the aforementioned\n configuration file and modify the paremeters you want to change (without deleting any\n  parameter). You can then use that config file with the following command:\n \n```\npython nilmth/train.py  --path_config \u003cpath to your config file\u003e\n```\n\nFor more information about the script, run:\n\n```\npython nilmth/train.py  --help\n```\n\nOnce the models are trained, test them with:\n\n```\npython nilmth/test.py  --path_config \u003cpath to your config file\u003e\n```\n\n### Reproduce the Paper\n\nTo reproduce the results shown in [our paper](#publications), activate the\n environment and then run:\n\n```\nnohup sh run/train_sequential.sh \u003e log.out \u0026 \n```\n\nThis will first create a folder named configs, where the different configurations of the\nmodels are stored. Then, the script `train.py` will be called, using each\n configuration each. This will store the model weights, which will be used\n again during the test phase:\n \n```\nnohup sh run/test_sequential.sh \u003e log.out \u0026 \n```\n\n### Thresholding Methods\n\nThere are three threshold methods available. Read [our paper](#publications)\nto understand how each threshold works.\n\n- 'mp', Middle-Point\n- 'vs', Variance-Sensitive\n- 'at', Activation Time\n\n## Publications\n\n* [NILM as a regression versus classification problem:\nthe importance of thresholding](https://www.researchgate.net/project/Non-Intrusive-Load-Monitoring-6)\n\n## Contact\n\nDaniel Precioso - [daniprec](https://github.com/daniprec) -  daniel.precioso@uca.es\n\nProject link: [https://github.com/UCA-Datalab/nilm-thresholding](https://github.com/UCA-Datalab/nilm-thresholding)\n\nResearhGate link: [https://www.researchgate.net/project/NILM-classification-VS-regression](https://www.researchgate.net/project/NILM-classification-VS-regression)\n\n## Acknowledgements\n\n* [UCA DataLab](http://datalab.uca.es/)\n* [David Gómez-Ullate](https://www.linkedin.com/in/david-g%C3%B3mez-ullate-oteiza-87a820b/?originalSubdomain=en)\n\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n[contributors-shield]: https://img.shields.io/github/contributors/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge\n[contributors-url]: https://github.com/UCA-Datalab/nilm-thresholding/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge\n[forks-url]: https://github.com/UCA-Datalab/nilm-thresholding/network/members\n[stars-shield]: https://img.shields.io/github/stars/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge\n[stars-url]: https://github.com/UCA-Datalab/nilm-thresholding/stargazers\n[issues-shield]: https://img.shields.io/github/issues/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge\n[issues-url]: https://github.com/UCA-Datalab/nilm-thresholding/issues\n[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge\u0026logo=linkedin\u0026colorB=555\n[linkedin-url]: https://www.linkedin.com/in/daniel-precioso-garcelan/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUCA-Datalab%2Fnilm-thresholding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FUCA-Datalab%2Fnilm-thresholding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUCA-Datalab%2Fnilm-thresholding/lists"}