Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/rob-med/awesome-TS-anomaly-detection

List of tools & datasets for anomaly detection on time-series data.
https://github.com/rob-med/awesome-TS-anomaly-detection

List: awesome-TS-anomaly-detection

anomaly-detection awesome-list data-analysis data-mining machine-learning outlier-detection temporal-data time-series

Last synced: 5 days ago
JSON representation

List of tools & datasets for anomaly detection on time-series data.

Host: GitHub
URL: https://github.com/rob-med/awesome-TS-anomaly-detection
Owner: rob-med
Created: 2017-12-19T15:05:20.000Z (almost 7 years ago)
Default Branch: master
Last Pushed: 2024-10-21T16:22:32.000Z (13 days ago)
Last Synced: 2024-10-29T17:55:38.275Z (5 days ago)
Topics: anomaly-detection, awesome-list, data-analysis, data-mining, machine-learning, outlier-detection, temporal-data, time-series
Homepage:
Size: 141 KB
Stars: 2,949
Watchers: 109
Forks: 450
Open Issues: 5
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

awesome-of-awesome-ml - awesome-TS-anomaly-detection (by rob-med)
awesome-time-series - List of tools & datasets for anomaly detection on time-series data
awesome-hrv - awesome-TS-anomaly-detection - series data. (Papers / Outiler)
Awesome-Paper-List - Anomaly Detection on Time-Series Data
awesome-sciml - rob-med/awesome-TS-anomaly-detection: List of tools & datasets for anomaly detection on time-series data.
awesome-machine-learning-resources - **[List - med/awesome-TS-anomaly-detection?style=social) (Table of Contents)
ultimate-awesome - awesome-TS-anomaly-detection - List of tools & datasets for anomaly detection on time-series data. (Programming Language Lists / Python Lists)
jimsghstars - rob-med/awesome-TS-anomaly-detection - List of tools & datasets for anomaly detection on time-series data. (Others)

README

        # awesome-TS-anomaly-detection

> List of tools & datasets for **anomaly detection on _time-series_ data**.

All lists are in alphabetical order. In the lists, maintaned projects are prioritized vs not mantained. 

A repository is considered "not maintained" if the latest commit is > 1 year old, or explicitly mentioned by the authors.

🎓 If you found this collection useful and want to cite it, please follow the [citation policy](#citation-policy).

## Anomaly Detection Software

| Name          | Language       | Pitch     | License | Maintained

| ------------- |:-------------: | :-------------: |  :-------------:  |  :-------------:     

| Cuebook's [CueObserve](https://github.com/cuebook/CueObserve)| Python3 | Anomaly detection on SQL data warehouses and databases. | Apache-2.0 |  :heavy_check_mark:

| Yahoo's [EGADS](https://github.com/yahoo/egads)                          | Java   |GADS is a library that contains a number of anomaly detection techniques applicable to many use-cases in a single package with the only dependency being Java.    | GPL | :heavy_check_mark:

| AIStream's [flow-forecast](https://github.com/AIStream-Peelout/flow-forecast)                          | Python   |  Deep learning PyTorch library for time series forecasting, classification, and anomaly detection (originally for flood forecasting).  | GPL-3 | :heavy_check_mark:

| [Hastic](https://github.com/hastic/hastic) | Python + node.js | Anomaly detection tool for time series data with Grafana-based UI.| GPL | :heavy_check_mark:

| Zillow's [Luminaire](https://github.com/zillow/luminaire)| Python | Luminaire is a python package that provides ML driven anomaly detection and forecasting solutions for time series data.  | Apache-2.0 |  :heavy_check_mark:

| [MIDAS](https://github.com/bhatiasiddharth/MIDAS) | C++ | MIDAS, short for Microcluster-Based Detector of Anomalies in Edge Streams, detects microcluster anomalies from an edge stream in constant time and memory. | Apache-2.0 | :heavy_check_mark:

| [Orion](https://github.com/sintel-dev/Orion)| Python | Orion is a machine learning library built for unsupervised time series anomaly detection, providing a number of “verified” ML pipelines (a.k.a Orion pipelines) that identify rare patterns and flag them for expert review. | MIT | :heavy_check_mark:

| [OutlierDetection.jl](https://github.com/OutlierDetectionJL/OutlierDetection.jl)| Julia | Fast, scalable and flexible Outlier Detection with Julia. | MIT | :heavy_check_mark:

| [PyOD](https://github.com/yzhao062/pyod)| Python | PyOD is a comprehensive and scalable Python toolkit for detecting outlying objects in multivariate data. | BSD 2-Clause |  :heavy_check_mark:

| [ruptures](https://github.com/deepcharles/ruptures/) | Python | Ruptures is a Python library for off-line change point detection. This package provides methods for the analysis and segmentation of non-stationary signals. | BSD 2-Clause |  :heavy_check_mark:

| EarthGecko [Skyline](https://github.com/earthgecko/skyline)              | Python3 | Skyline is a real-time anomaly detection system, built to enable passive monitoring of hundreds of thousands of metrics. | MIT | :heavy_check_mark:

| Expedia.com's [Adaptive Alerting](https://github.com/ExpediaDotCom/adaptive-alerting) | Java | Streaming anomaly detection with automated model selection and fitting. | Apache-2.0 | ❌

| Arundo's [ADTK](https://github.com/arundo/adtk) | Python | Anomaly Detection Toolkit (ADTK) is a Python package for unsupervised / rule-based time series anomaly detection. | MPL 2.0 | ❌

| Twitter's [AnomalyDetection](https://github.com/twitter/AnomalyDetection)| R      |AnomalyDetection is an open-source R package to detect anomalies which is robust, from a statistical standpoint, in the presence of seasonality and an underlying trend.   | GPL | ❌

| Lytics' [Anomalyzer](https://github.com/lytics/anomalyzer)                | Go     | Anomalyzer implements a suite of statistical tests that yield the probability that a given set of numeric input, typically a time series, contains anomalous behavior.    | Apache-2.0 | ❌

| [banpei](https://github.com/tsurubee/banpei)| Python | Outlier detection (Hotelling's theory) and Change point detection (Singular spectrum transformation) for time-series. | MIT | ❌

| Ele.me's [banshee](https://github.com/facesea/banshee)                     | Go     |Anomalies detection system for periodic metrics. | MIT | ❌

| [CAD](https://github.com/smirmik/CAD) | Python | Contextual Anomaly Detection for real-time AD on streagming data (winner algorithm of the 2016 NAB competition). | AGPL | ❌

| [Chaos Genius](https://github.com/chaos-genius/chaos_genius)| Python | ML powered analytics engine for outlier/anomaly detection and root cause analysis.  | MIT |  ❌

| Mentat's [datastream.io](https://github.com/MentatInnovations/datastream.io)| Python |An open-source framework for real-time anomaly detection using Python, Elasticsearch and Kibana. | Apache-2.0 |  ❌

| [DeepADoTS](https://github.com/KDD-OpenSource/DeepADoTS) | Python | Implementation and evaluation of 7 deep learning-based techniques for Anomaly Detection on Time-Series data. | MIT | ❌

| [Donut](https://github.com/korepwx/donut)| Python | Donut is an unsupervised anomaly detection algorithm for seasonal KPIs, based on Variational Autoencoders. | - | ❌

| [LoudML](https://github.com/regel/loudml)| Python | Loud ML is an open source time series inference engine built on top of TensorFlow. It's useful to forecast data, detect outliers, and automate your process using future knowledge.  | MIT |  ❌

| Linkedin's [luminol](https://github.com/linkedin/luminol)                | Python |Luminol is a light weight python library for time series data analysis. The two major functionalities it supports are anomaly detection and correlation. It can be used to investigate possible causes of anomaly. | Apache-2.0  | ❌ 

| Numenta's [Nupic](https://github.com/numenta/nupic)                      | C++    |Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM).   | AGPL |  ❌

| [oddstream](https://github.com/pridiltal/oddstream)| R | oddstream (Outlier Detection in Data Streams) provides real time support for early detection of anomalous series within a large collection of streaming time series data. | GPL-3 |  ❌

| [PyOdds](https://github.com/datamllab/pyodds)| Python | PyODDS is an end-to end Python system for outlier detection with database support. PyODDS provides outlier detection algorithms, which support both static and time-series data.  | MIT |  ❌

| [PySAD](https://github.com/selimfirat/pysad)| Python | PySAD is a streaming anomaly detection framework with various online models and complete set of tools for experimentation. | BSD 3-Clause |  ❌

| [rrcf](https://github.com/kLabUM/rrcf) | Python | Implementation of the Robust Random Cut Forest algorithm for anomaly detection on streams. | MIT | ❌

| Netflix's [Surus](https://github.com/netflix/surus)                      | Java   |Robust Anomaly Detection (RAD) - An implementation of the Robust PCA.   | Apache-2.0 | ❌ 

| NASA's [Telemanom](https://github.com/khundman/telemanom)| Python | A framework for using LSTMs to detect anomalies in multivariate time series data. Includes spacecraft anomaly data and experiments from the Mars Science Laboratory and SMAP missions.  | [custom](https://github.com/khundman/telemanom/blob/master/LICENSE.txt) |  ❌

## Related Software

This section includes some time-series software for anomaly detection-related tasks, such as forecasting, generic TS analysis and labeling.

### Forecasting

| Name          | Language       | Pitch     | License  | Maintained

| ------------- |:-------------: | :-------------: |  :-------------: |  :-------------: 

| [darts](https://github.com/unit8co/darts) | Python | darts is a python library for easy manipulation and forecasting of time series. It contains a variety of models, from classics such as ARIMA to neural networks. | Apache-2.0 |  :heavy_check_mark:

| [ETNA](https://github.com/tinkoff-ai/etna-ts) | Python | etna is a python library for time series forecasting and analysis with temporal data structure always in mind. Includes a variety of predictive models with unified interface along with EDA and validation methods. | Apache-2.0 |  :heavy_check_mark:

| Amazon's [GluonTS](https://github.com/awslabs/gluon-ts) | Python | GluonTS is a Python toolkit for probabilistic time series modeling, built around MXNet. GluonTS provides utilities for loading and iterating over time series datasets, state of the art models ready to be trained, and building blocks to define your own models. | Apache-2.0 |  :heavy_check_mark:

| [pmdarima](https://github.com/tgsmith61591/pyramid) | Python | Porting of R's _auto.arima_ with a scikit-learn-friendly interface. | MIT | :heavy_check_mark:

| Facebook's [Prophet](https://github.com/facebook/prophet) | Python/R | Prophet is a procedure for forecasting time series data. It is based on an additive model where non-linear trends are fit with yearly and weekly seasonality, plus holidays. | BSD |  :heavy_check_mark:

| [PyFlux](https://github.com/RJT1990/pyflux) | Python | The library has a good array of modern time series models, as well as a flexible array of inference options (frequentist and Bayesian) that can be applied to these models. | BSD 3-Clause | ❌

### Time-Series Analysis

| Name          | Language       | Pitch     | License  | Maintained

| ------------- |:-------------: | :-------------: |  :-------------: |  :-------------:

| Facebook's [Kats](https://github.com/facebookresearch/Kats/)| Python | Kats aims to provide the one-stop shop for time series analysis, including detection, forecasting, feature extraction/embedding, multivariate analysis, etc.  | MIT |  :heavy_check_mark:

| [MatrixProfile](https://github.com/matrix-profile-foundation/matrixprofile) | Python |  A Python 3 library making time series data mining tasks, utilizing matrix profile algorithms, accessible to everyone. | Apache-2.0 | :heavy_check_mark:

| Salesforce's [Merlion](https://github.com/salesforce/Merlion) | Python | Merlion: A Machine Learning Framework for Time Series Intelligence, support multiple models including ARIMA | BSD 3-Clause | :heavy_check_mark:

| [SaxPy](https://github.com/seninp/saxpy) | Python | General implementation of SAX, as well as HOTSAX for anomaly detection. | GPLv2.0 | :heavy_check_mark:

| [sktime](https://github.com/alan-turing-institute/sktime) | Python | A unified framework for machine learning with time series. It provides specialized time series algorithms and scikit-learn compatible tools to build, tune and validate time series models for multiple learning problems. | BSD 3-Clause | :heavy_check_mark:

| [sktime-dl](https://github.com/sktime/sktime-dl) | Python | An extension package for deep learning with Tensorflow/Keras for *sktime*. | BSD 3-Clause | :heavy_check_mark:

| [Squey](https://github.com/squey/squey) | C++ | Visualization software allowing in-depth exploration of timeseries while displaying every outliers. | MIT | :heavy_check_mark:

| [Tigramite](https://github.com/jakobrunge/tigramite) | Python | Tigramite is a causal time series analysis python package. It allows to efficiently reconstruct causal graphs from high-dimensional time series datasets and model the obtained causal dependencies for causal mediation and prediction analyses.| GPLv3.0 | :heavy_check_mark:

| [tsflex](https://github.com/predict-idlab/tsflex) | Python | tsflex is a time series toolkit for feature extraction & processing that is both flexible and efficient. This package supports strided-window feature extraction on multivariate, irregularly-sampled sequence data. | MIT | :heavy_check_mark:

| [tslearn](https://github.com/tslearn-team/tslearn) | Python | tslearn is a Python package that provides machine learning tools for the analysis of time series. This package builds on scikit-learn, numpy and scipy libraries. | BSD 2-Clause | :heavy_check_mark:

| [seglearn](https://github.com/dmbee/seglearn) | Python | Seglearn is a python package for machine learning time series or sequences. It provides an integrated pipeline for segmentation, feature extraction, feature processing, and final estimator. | BSD 3-Clause | ❌

### Labeling

| Name          | Language       | Pitch     | License | Maintained

| ------------- |:-------------: | :-------------: |  :-------------:  |  :-------------:

| Baidu's [Curve](https://github.com/baidu/Curve) | Python | Curve is an open-source tool to help label anomalies on time-series data. | Apache-2.0 | ❌

| Microsoft's [Taganomaly](https://github.com/Microsoft/TagAnomaly) | R (dockerized web app) | Simple tool for tagging time series data. Works for univariate and multivariate data, provides a reference anomaly prediction using Twitter's AnomalyDetection package. | MIT | ❌

## Benchmark Datasets

- Numenta's [NAB](https://github.com/numenta/NAB)

> NAB is a novel benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial timeseries data files plus a novel scoring mechanism designed for real-time applications.

- Yahoo's [Webscope S5](https://webscope.sandbox.yahoo.com/catalog.php?datatype=s&did=70)

> The dataset consists of real and synthetic time-series with tagged anomaly points. The dataset tests the detection accuracy of various anomaly-types including outliers and change-points. 

- 2020 Skoltech's [SKAB](https://github.com/waico/SkAB)

> SKAB (Skoltech Anomaly Benchmark) is designed for evaluating algorithms for anomaly detection. The benchmark currently includes 30+ datasets plus Python modules for algorithms’ evaluation. Each dataset represents a multivariate time series collected from the sensors installed on the testbed. All instances are labeled for evaluating the results of solving outlier detection and changepoint detection problems.

- 2018 AIOps's [KPI-Anomaly-Detection](https://github.com/NetManAIOps/KPI-Anomaly-Detection)

> The dataset consists of KPIs (key performace index) time series data from many real scenarios of Internet companies with ground truth label. Click below for a more detailed description.

  Details

  

  The dataset consists of KPI (key performace index) time series data from many real scenarios of Internet companies with ground truth label. KPIs fall into two broad categories: service KPIs and machine KPIs. Service KPIs are performance metrics that reflect the size and quality of a Web service, such as page response time, page views, and number of connection errors. Machine KPIs are performance indicators that reflect the health of the machine (server, router, switch), such as CPU utilization, memory utilization, disk IO, network card throughput, etc.

Dataset Descriptions:

In order to train the anomaly detection algorithm, the training KPI data provided by us is shown in table 1, including four columns: KPI ID, timestamp, the value of the KPI at that time, and whether the time is abnormal (label).

 Table 1 training KPI data case

 | KPI ID | Timestamp  | Value | Label |

 | ------ | ---------- | ----- | ----- |

 | 0      | 1503831000 | 10.8  | 0     |

 | 0      | 1503831060 | 12.3  | 1     |

 | ...    | ...        | ...   | ...   |

 In order to evaluate the anomaly detection algorithm, the test KPI data provided by us is shown in table 2, including two columns: KPI ID, timestamp, and the value corresponding to the KPI at that time.

 Table 2 testing KPI data case

 | KPI ID | Timestamp  | Value |

 | ------ | ---------- | ----- |

 | 0      | 1503831000 | 10.8  |

 | 0      | 1503831060 | 12.3  |

 | ...    | ...        | ...   |

## Citation Policy

If you would like to cite this repository, please use the following DOI: 

[![DOI](https://zenodo.org/badge/114778979.svg)](https://zenodo.org/badge/latestdoi/114778979)

or use this BibTex

```

 @article{medico2020, 

          title={rob-med/awesome-TS-anomaly-detection}, 

          DOI={10.5281/zenodo.3972944}, 

          abstractNote={A collection of tools and datasets for anomaly detection on time-series data.}, 

          publisher={Zenodo}, author={Roberto Medico}, year={2020}, month={Aug}}

```