Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/georgian-io/pyoats

Quick and Easy Time Series Outlier Detection
https://github.com/georgian-io/pyoats

anomaly anomaly-detection data-science deep-learning machine-learning time-series timeseries

Last synced: 14 days ago
JSON representation

Quick and Easy Time Series Outlier Detection

Awesome Lists containing this project

README 

        





[![Contributors][contributors-shield]][contributors-url]

[![Stargazers][stars-shield]][stars-url]

[![Issues][issues-shield]][issues-url]

[![Apache 2.0 License][license-shield]][license-url]

[![Last Commit][last_commit-shield]][last_commit-url]








  

    Logo

  



 OATS



  


    Quick and Easy Outlier Detection for Time Series 

    


    Explore the docs »

    


    


    View Demo

    ·

    Report Bug

    ·

    Request Feature

  





  Table of Contents

  
    

        
  1. 

          About The Project

          

        
    2. 

        
  2. 

          Getting Started

          

        
    3. 

        
  3. Usage
    4. 

        
  4. Models
    5. 

        
  5. Roadmap
    6. 

        
  6. Contributing
    7. 

        
  7. License
    8. 

        
  8. Contact
    9. 

        
  9. Acknowledgments
    10. 

      



## About The Project

Adapting existing outlier detection & prediction methods into a **time series outlier detection** system is not a simple task. Good news: **OATS** has done the heavy lifting for you! 



We present a straight-forward interface for popular, state-of-the-art detection methods to assist you in your experiments. In addition to the models, we also present different options when it comes to selecting a final threshold for predictions.



**OATS** seamlessly supports both univariate and multivariate time series regardless of the model choice and guarantees the same output shape, enabling a modular approach to time series anoamly detection.



(back to top)



### Built With

[![Python][Python.org]][Python-url] [![Poetry][Python-Poetry.org]][Poetry-url]



[![Pytorch][Pytorch.org]][Torch-url]  [![PytorchLightning][PytorchLightning.ai]][Lightning-url] [![TensorFlow][TensorFlow.org]][TF-url] [![Numpy][Numpy.org]][Numpy-url]



[![Darts][Darts]][Darts-url] [![PyOD][PyOD]][PyOD-url]



(back to top)



## Getting Started






    Usage Example

  

  



### Prerequisites

[![Python][Python.org]][Python-url] >=3.8, <3.11



#### For Docker Install:



[![Docker][Docker.com]][Docker-url]



#### For Local Install:



[![Poetry][Python-Poetry.org]][Poetry-url]



### Installation

#### PyPI

1. Install package via pip

   ```sh

   pip install pyoats

   ```

   **❗ Installing using an environment manager such as [`conda`](https://docs.conda.io/en/latest/miniconda.html), [`venv`](https://docs.python.org/3/library/venv.html), and [`poetry`](https://python-poetry.org/) is highly encouraged as this package contains deep learning frameworks.**

  

#### Docker

1. Clone the repo

    ```sh

    git clone https://github.com/georgian-io/pyoats.git && cd pyoats 

    ```

2. Build image

    ```sh

    docker build -t pyoats . 

    ```

3. Run Container

    ```sh 

    # CPU Only

    docker run -it pyoats

    

    # with GPU

    docker run -it --gpus all pyoats

    ```

    

#### Local

1. Clone the repo

    ```sh

    git clone https://github.com/georgian-io/pyoats.git && cd pyoats 

    ```

2. Install via Poetry

    ```sh 

    poetry install

    ```



(back to top)



## Usage



### Quick Start

For a quick start, please refer to our blog or copy our Colab notebook!



### Getting Anomaly Score

```python 

from oats.models import NHiTSModel



model = NHiTSModel(window=20, use_gpu=True)

model.fit(train)

scores = model.get_scores(test)

```

### Getting Threshold

```python 

from oats.threshold import QuantileThreshold



t = QuantileThreshold()

threshold = t.get_threshold(scores, 0.99)

anom = scores > threshold

```

_For more examples, please refer to the [Documentation](https://georgian-io.github.io/pyoats-docs/)_



(back to top)



## Models



_For more details about the individual models, please refer to the [Documentation](https://georgian-io.github.io/pyoats-docs/) or this blog for deeper explanation._



Model | Type | Multivariate Support* | Requires Fitting | DL Framework Dependency | Paper | Reference Model

--- | :---: | :---: | :---: | :---: | :---: |  :---: 

`ARIMA` | Predictive | ⚠️ | ✅ |  | | [`statsmodels.ARIMA`](https://www.statsmodels.org/dev/generated/statsmodels.tsa.arima.model.ARIMA.html)

`FluxEV` | Predictive | ⚠️ | ✅ |  | [📝](https://dl.acm.org/doi/10.1145/3437963.3441823) | 

`LightGBM` | Predictive | ⚠️ | ✅ |  |  | [`darts.LightGBM`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.gradient_boosted_model.html)

`Moving Average` | Predictive | ⚠️ |  |  |  | 

`N-BEATS` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | [📝](https://openreview.net/forum?id=r1ecqn4YwB) | [`darts.NBEATS`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nbeats.html)

`N-HiTS` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | [📝](https://arxiv.org/abs/2201.12886) | [`darts.NHiTS`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.nhits.html)

`RandomForest` | Predictive | ⚠️ | ✅ | | | [`darts.RandomForest`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.random_forest.html)

`Regression` | Predictive | ⚠️ | ✅ | | | [`darts.Regression`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.regression_model.html)

`RNN` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | | [`darts.RNN`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.rnn_model.html)

`Temporal Convolution Network` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | [📝](https://arxiv.org/abs/1803.01271) | [`darts.TCN`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tcn_model.html)

`Temporal Fusion Transformers` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | [📝](https://arxiv.org/abs/1912.09363) | [`darts.TFT`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.tft_model.html)

`Transformer` | Predictive | ✅ | ✅ | [![Pytorch][Pytorch.org]][Torch-url] | [📝](https://arxiv.org/abs/1706.03762) | [`darts.Transformer`](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.transformer_model.html)

`Isolation Forest` | Distance-Based | ✅ | ✅ | || [`pyod.IForest`](https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.iforest)

`Matrix Profile` | Distance-Based | ✅ |  | | [📝](https://www.cs.ucr.edu/~eamonn/MatrixProfile.html) | [`stumpy`](https://github.com/TDAmeritrade/stumpy)

`TranAD` | Reconstruction-Based | ✅ | ✅ | [![TensorFlow][TensorFlow.org]][Torch-url] | [📝](https://arxiv.org/abs/2201.07284) | [`tranad`](https://github.com/imperial-qore/TranAD)

`Variational Autoencoder` | Reconstruction-Based | ✅ | ✅ | [![TensorFlow][TensorFlow.org]][Torch-url] |   [📝](https://arxiv.org/abs/1312.6114) | [`pyod.VAE`](https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.vae)

`Quantile` | Rule-Based | ⚠️ |  |  || 



**\*** For models with ⚠️, score calculation is done separately along each column. This implicitly assumes independence of covariates, which means that **the resultant anomaly scores do not take into account of inter-variable dependency structures.**



(back to top)



## Roadmap



- [ ] Automatic hyper-parameter tuning

- [ ] More examples 

- [ ] More preprocessors

- [ ] More models from `pyod`



See the [open issues](https://github.com/georgian-io/pyoats/issues) for a full list of proposed features (and known issues).



(back to top)



## Contributing



Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.



If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".



Don't forget to give the project a star! Thanks again!



1. Fork the Project

2. Create your Feature Branch (`git checkout -b feature/amazing_feature`)

3. Commit your Changes (`git commit -m 'Add some amazing_feature'`)

4. Push to the Branch (`git push origin feature/amazing_feature`)

5. Open a Pull Request



(back to top)



## License



Distributed under the Apache 2.0 License. See `LICENSE` for more information.



(back to top)



## Contact





  

    Logo

  

  



|||||

|---|---|---|---|

| __Benjamin Ye__ | [![Github][BenGithub]][BenLinkedIn-url] | [![LinkedIn][BenLinkedIn]][BenLinkedIn-url] |  [![eMail][eMail]][BenEmail-url] 



Project Link: [https://github.com/georgian-io/oats](https://github.com/georgian-io/oats)



(back to top)



## Acknowledgments

I would like to thank my colleagues from Georgian for all the help and advice provided along the way.

* [Angeline Yasodhara](mailto:[email protected])

* [Akshay Budhkar](mailto:[email protected])

* [Borna Almasi](mailto:[email protected])

* [Parinaz Sobhani](mailto:[email protected])

* [Rodrigo Ceballos Lentini](mailto:[email protected])



I'd also like to extend my gratitude to all the contributors at [`Darts`][Darts-url] (for time series predictions) and [`PyOD`][PyOD-url] (for general outlier detection), whose projects have enabled a straight-forward extension into the domain of time series anomaly detection.



Finally, it'll be remiss of me to not mention [DATA Lab @ Rice University](https://cs.rice.edu/~xh37/index.html), whose wonderful [`TODS`][TODS-url] package served as a major inspiration for this project. Please check them out especially if you're looking for AutoML support.



[![Darts][Darts]][Darts-url] [![PyOD][PyOD]][PyOD-url] [![TODS][TODS]][TODS-url]



 

(back to top)



[contributors-shield]: https://img.shields.io/github/contributors/georgian-io/oats.svg?style=for-the-badge

[contributors-url]: https://github.com/georgian-io/pyoats/graphs/contributors

[forks-shield]: https://img.shields.io/github/forks/georgian-io/contributors.svg?style=for-the-badge

[forks-url]: https://github.com/georgian-io/pyoats/network/members

[stars-shield]: https://img.shields.io/github/stars/georgian-io/oats.svg?style=for-the-badge

[stars-url]: https://github.com/georgian-io/pyoats/stargazers

[issues-shield]: https://img.shields.io/github/issues/georgian-io/oats.svg?style=for-the-badge

[issues-url]: https://github.com/georgian-io/pyoats/issues

[license-shield]: https://img.shields.io/github/license/georgian-io/oats.svg?style=for-the-badge

[license-url]: https://github.com/georgian-io/pyoats/blob/master/LICENSE

[last_commit-shield]: https://img.shields.io/github/last-commit/georgian-io/oats.svg?style=for-the-badge

[last_commit-url]: https://github.com/georgian-io/pyoats/commits/



[Python-Poetry.org]: https://img.shields.io/badge/Poetry-60A5FA?style=for-the-badge&logo=poetry&logoColor=white

[Poetry-url]: https://www.python-poetry.org/

[Python.org]: https://img.shields.io/badge/Python-3776AB?style=for-the-badge&logo=python&logoColor=white

[Python-url]: https://www.python.org/

[PyTorch.org]: https://img.shields.io/badge/PyTorch-EE4C2C?style=for-the-badge&logo=pytorch&logoColor=white

[Torch-url]: https://pytorch.org/

[TensorFlow.org]: https://img.shields.io/badge/TensorFlow-FF6F00?style=for-the-badge&logo=tensorflow&logoColor=white

[TF-url]: https://www.tensorflow.org/

[Numpy.org]: https://img.shields.io/badge/Numpy-013243?style=for-the-badge&logo=numpy&logoColor=white

[Numpy-url]: https://www.numpy.org/

[Darts]: https://img.shields.io/badge/Repo-Darts-2100FF?style=for-the-badge&logo=github&logoColor=white

[Darts-url]: https://github.com/unit8co/darts

[PyOD]: https://img.shields.io/badge/Repo-PyOD-000000?style=for-the-badge&logo=github&logoColor=white

[PyOD-url]: https://github.com/yzhao062/pyod

[TODS]: https://img.shields.io/badge/Repo-TODS-29B48C?style=for-the-badge&logo=github&logoColor=white

[TODS-url]: https://github.com/datamllab/tods

[Docker.com]: https://img.shields.io/badge/Docker-2496ED?style=for-the-badge&logo=docker&logoColor=white

[Docker-url]: https://docker.com

[PyTorchLightning.ai]: https://img.shields.io/badge/lightning-792EE5?style=for-the-badge&logo=pytorchlightning&logoColor=white

[Lightning-url]: https://www.pytorchlightning.ai/



[BenLinkedIn]: https://img.shields.io/badge/LinkedIn-0A66C2?style=for-the-badge&logo=linkedin&logoColor=white

[BenLinkedIn-url]: https://www.linkedin.com/in/benjaminye/



[eMail]: https://img.shields.io/badge/EMail-EA4335?style=for-the-badge&logo=gmail&logoColor=white

[BenEmail-url]: mailto:[email protected]



[BenGithub]: https://img.shields.io/badge/Profile-14334A?style=for-the-badge&logo=github&logoColor=white

[BenGithub-url]: https://www.linkedin.com/in/benjaminye/