Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/UCA-Datalab/nilm-thresholding
:warning: This repository is no longer actively maintained. It previously dealt with Non-Intrusive Load Monitoring (NILM), focusing on predicting household appliance status from aggregated power load data. We explored different thresholding methods and evaluated deep learning models for regression and classification tasks.
https://github.com/UCA-Datalab/nilm-thresholding
convolutional-neural-networks deep-learning domestic-appliances gru monitoring neural-network nilm nilm-algorithms pytorch recurrent-neural-networks tensorflow thresholding thresholding-methods
Last synced: about 2 months ago
JSON representation
:warning: This repository is no longer actively maintained. It previously dealt with Non-Intrusive Load Monitoring (NILM), focusing on predicting household appliance status from aggregated power load data. We explored different thresholding methods and evaluated deep learning models for regression and classification tasks.
- Host: GitHub
- URL: https://github.com/UCA-Datalab/nilm-thresholding
- Owner: UCA-Datalab
- Created: 2020-04-29T14:54:25.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-09-14T09:33:37.000Z (about 1 year ago)
- Last Synced: 2024-02-14T21:24:02.117Z (7 months ago)
- Topics: convolutional-neural-networks, deep-learning, domestic-appliances, gru, monitoring, neural-network, nilm, nilm-algorithms, pytorch, recurrent-neural-networks, tensorflow, thresholding, thresholding-methods
- Language: Python
- Homepage: https://doi.org/10.1007/s11227-023-05149-8
- Size: 685 KB
- Stars: 33
- Watchers: 2
- Forks: 4
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![LinkedIn][linkedin-shield]][linkedin-url]
NILM: classification VS regression
:warning: **Notice:** This project is no longer actively maintained. Feel free to use the existing code, but please note that there won't be further updates or bug fixes.
Table of Contents
-
About The Project
-
Getting Started
-
Datasets
- Preprocess the Data
-
Train
- Publications
- Contact
- Acknowledgements
## About the project
Non-Intrusive Load Monitoring (NILM) aims to predict the status
or consumption of domestic appliances in a household only by knowing
the aggregated power load. NILM can be formulated as regression problem
or most often as a classification problem. Most datasets gathered
by smart meters allow to define naturally a regression problem,
but the corresponding classification problem is a derived one,
since it requires a conversion from the power signal to the status of each
device by a thresholding method. We treat three different thresholding
methods to perform this task, discussing their differences on various
devices from the UK-DALE dataset. We analyze the performance of
deep learning state-of-the-art architectures on both the regression and
classification problems, introducing criteria to select the most convenient
thresholding method.
## Getting started
### Create the Environment
To create the environment using Conda:
1. Install miniconda
```
curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh | bash
```
Say yes to everything and accept default locations. Refresh bash shell with `bash -l`
2. Update conda
```
conda update -n base -c defaults conda
```
3. Clone this repository and cd into the folder
4. Create and activate conda environment (removing previously existing env of the same name)
```
conda remove --name nilm-thresholding --all
conda env create -f environment.yml --force
conda activate nilm-thresholding
```
## Datasets
### UK-DALE
UK-DALE dataset is hosted on the following link:
[https://data.ukedc.rl.ac.uk/browse/edc/efficiency/residential
/EnergyConsumption/Domestic/UK-DALE-2017/UK-DALE-FULL-disaggregated](https://data.ukedc.rl.ac.uk/browse/edc/efficiency/residential/EnergyConsumption/Domestic/UK-DALE-2017/UK-DALE-FULL-disaggregated)
The files needed by this module are *ukdale.zip* and *ukdale.h5.zip*.
Download both and unzip them in a folder named [data](/data) inside the root.
Once you are done, your local directory should look like this:
```
nilm-thresholding
|_ nilmth
|_ [python scripts and subfolders]
|_ data
|_ ukdale
|_ [house_1 to house_5]
|_ ukdale.h5
```
Credit: [Jack Kelly](https://jack-kelly.com/data/)
### Pecan Street Dataport
We are aiming to include this dataset in a future release. You can check the issue here: [https://github.com/UCA-Datalab/nilm-thresholding/issues/8](https://github.com/UCA-Datalab/nilm-thresholding/issues/8)
Any help and suggestions are welcome!
Credit: [Pecan Street](https://dataport.pecanstreet.org/)
## Preprocess the Data
Once downloaded the raw data from any of the sources above,
you must preprocess it.
This is done by running the following script:
```
python nilmth/preprocess.py
```
This will generate a new folder, named 'data-prep',
containing all the preprocessed data.
## Train
The folder [nilmth](/nilmth) contains an executable script to train the
models. Run the following line on the root folder
(make sure to have the enviroment active and the data downloaded):
```
python nilmth/train.py
```
This will train the CONV model using the default parameters.
The script stores several outputs in the [outputs folder](/outputs),
including:
- .txt files with the model scores over the validation data
- .pth files containing the model weights
- .png files with a section of the validation data and the model's prediction
The list with all the available parameters and their default values is stored in the
[configuration file](nilmth/config.toml).
If you want to use your own set of parameters, duplicate the aforementioned
configuration file and modify the paremeters you want to change (without deleting any
parameter). You can then use that config file with the following command:
```
python nilmth/train.py --path_config
```
For more information about the script, run:
```
python nilmth/train.py --help
```
Once the models are trained, test them with:
```
python nilmth/test.py --path_config
```
### Reproduce the Paper
To reproduce the results shown in [our paper](#publications), activate the
environment and then run:
```
nohup sh run/train_sequential.sh > log.out &
```
This will first create a folder named configs, where the different configurations of the
models are stored. Then, the script `train.py` will be called, using each
configuration each. This will store the model weights, which will be used
again during the test phase:
```
nohup sh run/test_sequential.sh > log.out &
```
### Thresholding Methods
There are three threshold methods available. Read [our paper](#publications)
to understand how each threshold works.
- 'mp', Middle-Point
- 'vs', Variance-Sensitive
- 'at', Activation Time
## Publications
* [NILM as a regression versus classification problem:
the importance of thresholding](https://www.researchgate.net/project/Non-Intrusive-Load-Monitoring-6)
## Contact
Daniel Precioso - [daniprec](https://github.com/daniprec) - [email protected]
Project link: [https://github.com/UCA-Datalab/nilm-thresholding](https://github.com/UCA-Datalab/nilm-thresholding)
ResearhGate link: [https://www.researchgate.net/project/NILM-classification-VS-regression](https://www.researchgate.net/project/NILM-classification-VS-regression)
## Acknowledgements
* [UCA DataLab](http://datalab.uca.es/)
* [David Gómez-Ullate](https://www.linkedin.com/in/david-g%C3%B3mez-ullate-oteiza-87a820b/?originalSubdomain=en)
[contributors-shield]: https://img.shields.io/github/contributors/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge
[contributors-url]: https://github.com/UCA-Datalab/nilm-thresholding/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge
[forks-url]: https://github.com/UCA-Datalab/nilm-thresholding/network/members
[stars-shield]: https://img.shields.io/github/stars/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge
[stars-url]: https://github.com/UCA-Datalab/nilm-thresholding/stargazers
[issues-shield]: https://img.shields.io/github/issues/UCA-Datalab/nilm-thresholding.svg?style=for-the-badge
[issues-url]: https://github.com/UCA-Datalab/nilm-thresholding/issues
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
[linkedin-url]: https://www.linkedin.com/in/daniel-precioso-garcelan/