https://github.com/ftisiot/anomaly-detection-with-apache-flink-and-kafka

Anomaly detection demo with Apache Flink® and Apache Kafka®
https://github.com/ftisiot/anomaly-detection-with-apache-flink-and-kafka

Last synced: about 16 hours ago
JSON representation

Anomaly detection demo with Apache Flink® and Apache Kafka®

• Host: GitHub
• URL: https://github.com/ftisiot/anomaly-detection-with-apache-flink-and-kafka
• Owner: ftisiot
• License: apache-2.0
• Created: 2022-08-02T10:24:29.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-11-07T13:04:11.000Z (about 1 year ago)
• Last Synced: 2023-11-07T14:24:20.205Z (about 1 year ago)
• Language: Shell
• Size: 38.3 MB
• Stars: 4
• Watchers: 1
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Security: SECURITY.md

Awesome Lists containing this project

README

        
Streaming Anomaly Detection with Apache Flink® and Apache Kafka®

================================================================

This is demo repository showing how to create a streaming anomaly detection system using Apache Kafka® and Apache Flink®.

Overview

========

The repository shows a silly pizza example, based on a python [Fake data producer](https://github.com/aiven/fake-data-producer-for-apache-kafka-docker)

Features

============

The example demonstrates how:

* [Create an Aiven for Apache Kafka® and Aiven for Apache Flink® services](demo.md#create-services). This step is strictly not necessary you can use any Apache Kafka and Apache Flink instance. The anomaly detection SQL queries will work over any Apache Flink® 1.14+ version.

* [Create a Fake Pizza streaming data generator using docker](demo.md#start-streaming-pizzas)

* [Create an Apache Flink SQL job for basic filtering](demo.md#apply-basic-filtering)

* [Create an Apache Flink SQL job for basic aggregation](demo.md#aggregating-data)

* [Create an Apache Flink SQL job to recognise patterns](demo.md#check-for-trends)

Setup

============

To be able to run the scripts you need:

* The [Aiven CLI](https://developer.aiven.io/docs/tools/cli.html) installed and an [Aiven valid login token](https://console.aiven.io/signup)

* Docker installed

The complete set of steps is available in the [demo.md](demo.md) file.

License

============

Streaming Anomaly Detection with Apache Flink® and Apache Kafka® is licensed under the Apache license, version 2.0. Full license text is available in the [LICENSE](LICENSE) file.

Please note that the project explicitly does not require a CLA (Contributor License Agreement) from its contributors.

Contact

============

Bug reports and patches are very welcome, please post them as GitHub issues and pull requests at https://github.com/aiven/anomaly-detection-with-apache-flink-and-kafka. 

To report any possible vulnerabilities or other serious issues please see our [security](SECURITY.md) policy.