{"id":13474159,"url":"https://github.com/KIT-IAI/pyWATTS","last_synced_at":"2025-03-26T21:30:39.178Z","repository":{"id":39790263,"uuid":"297579218","full_name":"KIT-IAI/pyWATTS","owner":"KIT-IAI","description":"pyWATTS: Python Workflow Automation Tool for Time-Series","archived":true,"fork":false,"pushed_at":"2024-06-22T06:57:29.000Z","size":776,"stargazers_count":41,"open_issues_count":17,"forks_count":8,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-08-01T16:35:01.286Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KIT-IAI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-22T08:06:28.000Z","updated_at":"2024-07-22T15:44:19.000Z","dependencies_parsed_at":"2024-01-12T17:34:43.276Z","dependency_job_id":"9c6e58e5-28ab-4a3e-9a70-0ed9c6afa4d7","html_url":"https://github.com/KIT-IAI/pyWATTS","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KIT-IAI%2FpyWATTS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KIT-IAI%2FpyWATTS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KIT-IAI%2FpyWATTS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KIT-IAI%2FpyWATTS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KIT-IAI","download_url":"https://codeload.github.com/KIT-IAI/pyWATTS/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222169239,"owners_count":16942668,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T16:01:09.957Z","updated_at":"2024-10-30T05:30:38.619Z","avatar_url":"https://github.com/KIT-IAI.png","language":"Python","funding_links":[],"categories":["Python","Code-Resource"],"sub_categories":["2020"],"readme":"![Pipeline status](https://github.com/KIT-IAI/pyWATTS/workflows/Python%20application/badge.svg?branch=master)\n[![DOI](https://zenodo.org/badge/297579218.svg)](https://zenodo.org/badge/latestdoi/297579218)\n[![Documentation](https://readthedocs.org/projects/pywatts/badge/)](https://pywatts.readthedocs.io/en/latest/)\n\n\n# pyWATTS - Now Deprecated\n\n**Please note that pyWATTS is no longer actively maintained. Therefore, the contents of this repository are now read-only!**\n\n**The graph pipeline functionality of pyWATTS has now been integrated into the open source python package [sktime](https://www.sktime.net/en/stable/).**\n\nWe would like to thank everybody who has helped develop, test, and use pyWATTS in the last few years and strongly advise all past pyWATTS users to apply [sktime](https://www.sktime.net/en/stable/) in the future.\n\n\n# Legacy Instructions for Use\n\nIf you still wish to use pyWATTS you can follow the legacy instructions below.\n\n## Installation\n\nTo install this project, perform the following steps.\n\n1. Clone the project\n2. Open a terminal of the virtual environment where you want to use the project\n2. cd pywatts\n3. ``pip install .`` or ``pip install -e .``\n   if you want to install the project editable. If you aim to develop code for pyWATTS, you should\n   use:  ``pip install -e .[dev]``\n\n### TensorFlow for Mac\nIf you want to use pyWATTS on an Apple computer, you currently have to install TensorFlow separately:\n1. Install Apple TensorFlow dependencies: ``conda install -c apple tensorflow-deps``\n2. Install TensorFlow for macOS: ``pip install tensorflow-macos``\n3. Install TensorFlow metal for GPU usage: ``pip install tensorflow-metal``\n\nMore information regarding TensorFlow for macOS can be found through this helpful [installation guide](https://caffeinedev.medium.com/how-to-install-tensorflow-on-m1-mac-8e9b91d93706).\n\n---\n**NOTE**\nIf you want to use torch, you have to install it by yourself, since it is not possible to install torch via pypi on\nwindows. To install torch take a look at\n[install Pytorch](https://pytorch.org/get-started/locally/).\n\n---\n\n## How to use\n\nA simple example is given in example.py. If you like a more detailed explanation of this example, look at the\ngetting_started page in our documentations.\n\n1. You need to create a pipeline.\n2. You can add modules to the pipeline by calling them with either the previous step or with the pipeline, if it is a\n   start step (i.e. Functional API).\n3. Run the pipeline with pipeline.run()\n\nAn extract of the example in example.py is given in the following:\n\n```python\nfrom pywatts.core.pipeline import Pipeline\nfrom pywatts.modules import SKLearnWrapper\nfrom pywatts.modules import LinearInterpolater, CalendarFeature, CalendarExtraction\nfrom sklearn.preprocessing import StandardScaler\n\npipeline = Pipeline()\n\n# Add modules\ncalendar = CalendarExtraction(continent=\"Europe\", country=\"Germany\", features=[CalendarFeature.month,\n                                                                               CalendarFeature.weekday,\n                                                                               CalendarFeature.weekend]\n                              )(x=pipeline[\"load_power_statistics\"])\n\n# Deal with missing values through linear interpolation\nimputer_power_statistics = LinearInterpolater(\n    method=\"nearest\", dim=\"time\", name=\"imputer_power\"\n)(x=pipeline[\"load_power_statistics\"])\n\npower_scaler = SKLearnWrapper(module=StandardScaler(), name=\"scaler_power\")(x=imputer_power_statistics)\n\n# Train the pipeline\npipeline.train(\"data/getting_started_data.csv\")\n```\n\n## Goals\n\nThe goals of pyWATTS (Python Workflow Automation Tool for Time-Series) are\n\n* to support researchers in conducting automated time series experiments independent of the execution environment and\n* to make methods developed during the research easily reusable for other researchers.\n\nTherefore, pyWATTS is an automation tool for time series analysis that implements three core ideas:\n\n* pyWATTS provides a pipeline to support the execution of experiments. This way, the execution of simple and often\n  recurring tasks is simplified. For example, a defined preprocessing pipeline could be reused in other experiments.\n  Furthermore, the execution of defined pipelines is independent of the execution environment. Consequently, for the\n  repetition or reuse of a third-party experiment or pipeline, it should be sufficient to install pyWATTS and clone the\n  third-party repository.\n* pyWATTS allows to define end-to-end pipelines for experiments. Therefore, experiments can be easily executed that\n  comprise the preprocessing, models and benchmark training, evaluation, and comparison of the models with the\n  benchmark.\n* pyWATTS defines an API that forces the different methods (called modules) to have the same interface in order to make\n  newly developed methods more reusable.\n\n## Features\n\n* Reuseable modules\n* Plug-and-play architecture to insert modules into the pipeline\n* End-to-end pipeline for experiments such that pipeline performs all necessary steps from preprocessing to evaluation\n* Conditions within the pipeline\n* Saving and loading of the entire pipeline including the pipeline modules\n* Adapters and wrappers for existing machine learning libraries\n\n## Programming Guidelines\n\n* Implement new features on a new, separate branch (see \"Module Implementation Worflow\" below). If everything works,\n  open a pull request to merge the branch with the master. Note that you should name your branch with the following\n  convention: \u003cfeature|docs|bugfix\u003e/\u003cissue_number\u003e_\u003cdescriptive_name\u003e\n* Provide tests for your module (see \"Tests\" below).\n* Provide proper logging information (see \"Logging\" below).\n* Use a linter, follow pep8, and add docstrings to your classes and methods\u003cbr\u003e\n  To do so in PyCharm, activate in Settings -\u003e Editor -\u003e Inspections -\u003e Python:\n    * \"PEP8 coding style violation\"\n    * \"missing or empty docstring\"\n    * \"missing type hinting for function parameter\"\n    * \"package requirement\"\n* Use typing (see https://docs.python.org/3/library/typing.html).\n\n## Module Implementation Workflow\n\n* Create an issue or choose an existing one and assign yourself to the issue. This way, everyone knows whether an issue\n  is under development or not.\n* Create a new branch for your module/feature. For naming your branch use the following naming convention \u003c\n  feature|docs|bugfix\u003e/\u003cissue_number\u003e_\u003cdescriptive_name\u003e.\n* Decide whether your module is a Transformer or an Estimator. Inherit of the BaseEstimator or BaseTransformer\n  accordingly.\n\n  | Estimator   | Transformer   |\n    |-----|--------|\n  | An Estimator is everything that is fitted before the prediction | A transformer converts an input to an output, it does not need to be fitted before.\n\n* Implement the abstract methods of either BaseTransformer or BaseEstimator. The folllowing abstract methods exist:\n\n  | Method          | Base Estimator | BaseTransformer |\n    |-----------------|----------------|-----------------|\n  | getParam        | yes            | yes             |\n  | setParam        | yes            | yes             |\n  | fit             | yes            | no              |\n  | transform       | yes            | yes             |\n\n* Add test cases for your module/feature. For example, if you implement a module, add the tests in the folder\n  tests/unit/modules.\n* Add a documentation for your module/feature\n    * Use a docstring for your class (i.e. the module you developed) to describe what your class does and to describe\n      the parameters\n    * Use a docstring for each method\n* Commit and push your changes\n* Open a merge request and assign a maintainer\n* If all tests are green, your code works, and your code is documented, then the assigned maintainer will merge your\n  module/feature.\n\n## Tests\n\n### Writing tests\n\nTo write tests, see the test template where the different methods are explained. When writing tests, please consider the\nfollowing basic rules for testing:\n\n* In general, tests should not only cover the normal case but also edge cases such as exceptions when wrong inputs are\n  used.\n* After writing tests, please also check if the tests sufficiently cover the code of your module. However, be aware that\n  the line-coverage metric provided will only help you to identify uncovered source code. Please also note that 100%\n  line coverage does not mean that your code is free of bugs.\n* If you fix a bug, implement a test case which repeats the bug.\n\nIf you are looking for some test guidelines, have a look at https://docs.python-guide.org/writing/tests/\n\n### Test libraries\n\n* **[unittest](https://docs.python.org/3/library/unittest.html)** We use unittests to define our test cases.\n    * **[unittest.mock](https://docs.python.org/3/library/unittest.mock.html)** We use this mock object library to mock\n      calls of other methods. In mocks, it is possible to check whether a method is called correctly.\n    * **[unittest.mock.patch](https://docs.python.org/3/library/unittest.mock.html#the-patchers)** We use this library\n      to replace objects imported in the module under test with mock objects.\n* **[pytest](https://docs.pytest.org/en/latest/)** We use pytest to run our tests.\n* **[pytest-cov](https://pytest-cov.readthedocs.io/en/latest/)** We use pytest-coverage to calculate the test coverage\n  of our source code.\n\n### Run tests\n\nTo run tests, use your preferred editor, IDE, or the following CLI command\n\n``\npytest tests\n``\n\nThis command executes all tests and prints the results. It also provides the code coverage of the project.\n\n## Documentation\n\nTo build the documentation, we use Sphinx. Perhaps, you first need to install it via pip.\n\n### Documentation of the API\n\nWe use Sphinx-apidoc to automatically generate the documentation from the Python docstrings. Therefore, it is necessary\nthat each module, class, and method is documented with such a string. The source files of the API documentation are\nlocated in docs/source/api. Do not change the files in this directory because they are automatically generated and\nconsequently the generation process would override any changes.\n\n### Extending the documentation\n\nTo extend the documentation, you have to create a new restructured text (RST) file in docs/source. This file has to be\nincluded in the index.rst or in another part of the documentation.\n\n### Building the documentation\n\nThere are mainly two steps to build the documentation.\n\n1. Create the documentation of the API. This is done via\n   `sphinx-apidoc -o docs/source/api pywatts`. This command creates the rst files from the docstrings.\n2. Build the documentation as html. For this, you have to change to the docs directory `cd docs` and execute `make html`\n   . This command generates the static html website out of the rst files.\n\nTo view the documentation website locally, open the index.html in docs/build/html in your webbrowser.\n\n### Further information on restructured text\n\n* [RST Cheatsheet](https://github.com/ralsina/rst-cheatsheet/blob/master/rst-cheatsheet.rst)\n\n## Logging\n\nWe use logging to obtain information from the pipeline about\n\n* Modules added to the pipeline and\n* Calls of fit and transform on modules.\n\nTherefore, use the logging library provided by the Python standard library. Currently, the log information is logged\ninto pywatts.log.\n\n**Note** Try to avoid print statements for messages. Instead, use logging with an appropriate logging level.\n\n### Further information on logging\n\n* [Basic logging tutorial](https://docs.python.org/3/howto/logging.html#basic-logging-tutorial)\n* [Logging Cookbook](https://docs.python.org/3/howto/logging-cookbook.html#logging-cookbook)\n\n## Glossary\n\nFor a common understanding of the various terms used in this framework, see the following table:\n\n| Term            | Explanation                      |\n  |-----------------|----------------------------------|\n| Pipeline        | A pipeline implements the workflow. It executes the fit and transform methods of the individual modules in the pipeline.|\n| Module          | A module is an element of the pipeline. Each module implements only one specific task, such as the detection of missing values or a clock shift.|\n| BaseEstimator   | A module that has to be fitted must inherit from BaseEstimator. Such a module is also called Estimator.|\n| BaseTransformer | A module that only transforms data must inherit from BaseTransformer. Such a module is also called Transformer.|\n| Wrapper         | A wrapper is a special type of module that wraps models and methods of external libraries, such as sklearn.|  \n| Summary         | A summary is an element of the pipeline. In contrast to the module it calculates one value for summarizing the time series instead of transforming it into a new.\n| Step            | A step manages the execution of a single module, such as fetching the input, checking condition, and providing outputs.\n\n## Current Development Status\n\n**PyWATTS is no longer maintained and therefore deprecated**\n\n# Funding\n\nThis project is supported by the Helmholtz Association under the Program “Energy System Design”, by the Helmholtz Association’s Initiative and Networking Fund through Helmholtz AI, by the Helmholtz Association under the Joint Initiative \"Energy System 2050 - A Contribution of the Research Field Energy\", and by the German Research Foundation (DFG) Research Training Group 2153 \"Energy Status Data: Informatics Methods for its Collection, Analysis and Exploitation\".\n\n# Citation\nIf you use this framework in a scientific publication please cite the corresponding paper:\n\n\u003eBenedikt Heidrich, Andreas Bartschat, Marian Turowski, Oliver Neumann, Kaleb Phipps, Stefan Meisenbacher, Kai Schmieder, Nicole Ludwig, Ralf Mikut, Veit Hagenmeyer. “pyWATTS: Python Workflow Automation Tool for Time Series.” (2021). ). arXiv:2106.10157. http://arxiv.org/abs/2106.10157\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKIT-IAI%2FpyWATTS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKIT-IAI%2FpyWATTS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKIT-IAI%2FpyWATTS/lists"}