Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mbode/flink-prometheus-example

Example setup to demonstrate Prometheus integration of Apache Flink
https://github.com/mbode/flink-prometheus-example

flink prometheus

Last synced: about 2 months ago
JSON representation

Example setup to demonstrate Prometheus integration of Apache Flink

Host: GitHub
URL: https://github.com/mbode/flink-prometheus-example
Owner: mbode
License: apache-2.0
Created: 2017-10-20T08:12:31.000Z (about 7 years ago)
Default Branch: master
Last Pushed: 2024-04-12T06:24:26.000Z (9 months ago)
Last Synced: 2024-04-12T15:08:07.376Z (9 months ago)
Topics: flink, prometheus
Language: Java
Homepage:
Size: 733 KB
Stars: 88
Watchers: 5
Forks: 37
Open Issues: 2
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        [![Actions Status](https://github.com/mbode/flink-prometheus-example/workflows/Gradle/badge.svg)](https://github.com/mbode/flink-prometheus-example/actions)

[![codecov](https://codecov.io/gh/mbode/flink-prometheus-example/branch/master/graph/badge.svg)](https://codecov.io/gh/mbode/flink-prometheus-example)

[![Flink v1.20.0](https://img.shields.io/badge/flink-v1.20.0-blue.svg)](https://github.com/apache/flink/releases/tag/release-1.20.0)

[![Prometheus v2.37.1](https://img.shields.io/badge/prometheus-v2.37.1-blue.svg)](https://github.com/prometheus/prometheus/releases/tag/v2.37.1)

This repository contains the live demo to my talk _Monitoring Flink with Prometheus_, which I have given at:

* [Flink Forward Berlin 2018](https://berlin-2018.flink-forward.org/conference-program/#monitoring-flink-with-prometheus), _2018-09-04_ (:video_camera: [Video](https://www.youtube.com/watch?v=vesj-ghLimA) :page_facing_up: [Slides](https://www.slideshare.net/MaximilianBode1/monitoring-flink-with-prometheus))

* [Spark & Hadoop User Group Munich](https://www.meetup.com/de-DE/Hadoop-User-Group-Munich/events/252393503/), _2018-09-26_

The blog post [Flink and Prometheus: Cloud-native monitoring of streaming applications](https://flink.apache.org/features/2019/03/11/prometheus-monitoring.html) explains how to run the demo yourself.

## Getting Started

### Startup

```

./gradlew composeUp

```

### Web UIs

- [Flink JobManager](http://localhost:8081/#/overview)

- [Prometheus](http://localhost:9090/graph)

- [Grafana](http://localhost:3000) (credentials _admin:flink_)

- Prometheus endpoints

    - [Job Manager](http://localhost:9249/metrics)

    - [Task Manager 1](http://localhost:9250/metrics)

    - [Task Manager 2](http://localhost:9251/metrics)

## Built With

- [Apache Flink](https://flink.apache.org)

- [Prometheus](https://prometheus.io)

- [Grafana](https://grafana.com)

- [docker-compose](https://docs.docker.com/compose/) – provisioning of the test environment

- [Gradle](https://gradle.org) with [kotlin-dsl](https://github.com/gradle/kotlin-dsl)

    - [gradle-testsets-plugin](https://github.com/unbroken-dome/gradle-testsets-plugin)

    - [shadow](https://github.com/johnrengelman/shadow)

    - [spotless](https://github.com/diffplug/spotless/tree/master/plugin-gradle)

    - [spotbugs](https://github.com/spotbugs/spotbugs-gradle-plugin)

    - [gradle-docker-compose-plugin](https://github.com/avast/gradle-docker-compose-plugin)

    - [gradle-versions-plugin](https://github.com/ben-manes/gradle-versions-plugin)

## Development

typical tasks:

- verify: `./gradlew check`

- integration tests: `./gradlew integrationTest`

- list outdated dependenices: `./gradlew dependencyUpdates`

- update gradle: `./gradlew wrapper --gradle-version=` (twice)