Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kuzzleio/kuzzle-plugin-prometheus
Monitor your Kuzzle application with Prometheus
https://github.com/kuzzleio/kuzzle-plugin-prometheus
devops hacktoberfest kuzzle kuzzle-plugin monitoring prometheus
Last synced: about 2 months ago
JSON representation
Monitor your Kuzzle application with Prometheus
- Host: GitHub
- URL: https://github.com/kuzzleio/kuzzle-plugin-prometheus
- Owner: kuzzleio
- License: apache-2.0
- Created: 2019-05-23T16:17:15.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-02-01T14:39:59.000Z (8 months ago)
- Last Synced: 2024-07-02T00:50:17.550Z (3 months ago)
- Topics: devops, hacktoberfest, kuzzle, kuzzle-plugin, monitoring, prometheus
- Language: TypeScript
- Homepage:
- Size: 2.58 MB
- Stars: 4
- Watchers: 11
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
- [About](#about)
- [Kuzzle Prometheus Plugin](#kuzzle-prometheus-plugin)
- [Kuzzle](#kuzzle)
- [Compatibility matrix](#compatibility-matrix)
- [Installation](#installation)
- [Configuration](#configuration)
- [Plugin](#plugin)
- [Prometheus](#prometheus)
- [With only one Kuzzle node](#with-only-one-kuzzle-node)
- [With an authentified user](#with-an-authentified-user)
- [With multiple Kuzzle nodes and using Docker Compose](#with-multiple-kuzzle-nodes-and-using-docker-compose)
- [Using Kubernetes annotations](#using-kubernetes-annotations)
- [Dashboards](#dashboards)
- [Features](#features)
- [Screenshots](#screenshots)
- [Local development](#local-development)
- [Migrations](#migrations)
- [From version 3.x to 4.x](#from-version-3x-to-4x)
- [Migration steps](#migration-steps)
# About
## Kuzzle Prometheus Plugin
This is the official Prometheus monitoring plugin for the free and open-source backend Kuzzle.
It provides you features such as:
* [Kuzzle cluster mode](https://github.com/kuzzleio/kuzzle-plugin-cluster) compatibility.
* Event based monitoring using [Kuzzle Events](https://docs.kuzzle.io/core/1/plugins/guides/events/intro/).
* System metrics (CPU, RAM, I/O...).
## Kuzzle
Kuzzle is an open-source backend that includes a scalable server, a multiprotocol API,
an administration console and a set of plugins that provide advanced functionalities like real-time pub/sub, blazing fast search and geofencing.
* :octocat: __[Github](https://github.com/kuzzleio/kuzzle)__
* :earth_africa: __[Website](https://kuzzle.io)__
* :books: __[Documentation](https://docs.kuzzle.io)__
* :email: __[Gitter](https://gitter.im/kuzzleio/kuzzle)__
## Compatibility matrix
| Kuzzle Version | Plugin Version |
| -------------- | -------------- |
| 1.10.x | 1.x.x |
| 2.x.x | 2.x.x |
| 3.x.x | >= 2.11.x |
| 4.x.x | >= 2.16.9 |
# Installation
To install this plugin on your Kuzzle stack (for each of your Kuzzle nodes),
you will first need a Kuzzle Application like so. (see [Getting Started](/core/2/guides/getting-started))
```typescript
import { Backend } from 'kuzzle';
const app = new Backend('kuzzle');
app.start()
.then(() => {
app.log.info('Application started');
})
.catch(console.error);
```
Once you have it, you will need to:
- Import the Prometheus plugin,
- Create a new instance of the plugin
- And then use it in your application.
You will end up with something like this:
```typescript
import { Backend } from 'kuzzle';
import { PrometheusPlugin } from 'kuzzle-plugin-prometheus'; // Import the prometheus plugin
const app = new Backend('kuzzle');
const prometheusPlugin = new PrometheusPlugin(); // Create a new instance of the prometheus plugin
app.plugin.use(prometheusPlugin); // Add the plugin to your application
app.start()
.then(() => {
app.log.info('Application started');
})
.catch(console.error);
```
# Configuration
You can find sample configuration files for this plugin and the Prometheus scraping job in the `demo` folder.
## Plugin
This plugin is configurable using the `kuzzlerc` Kuzzle configuration file.
```json
{
"plugins": {
"prometheus": {
"default": {
"enabled": true,
"prefix": "",
"eventLoopMonitoringPrecision": 10,
"gcDurationBuckets": [0.001, 0.01, 0.1, 1, 2, 5]
},
"core": {
"monitorRequestDuration": true,
"prefix": "kuzzle_"
},
"labels": {
"project": "mySuperProject",
"environment": "development"
}
}
}
}
```
* `default`: Default Node.js metrics retrieved by [the Prom Client library](https://github.com/siimon/prom-client/tree/master/lib/metrics)
* `enabled`: Enable/Disable the default Node.js metrics (default: `true`)
* `prefix`: String to use to prefix metrics name (default: an empty string to avoid conflicts when using official Grafana dashboards)
* `eventLoopMonitoringPrecision`: Node.js Event Loop sampling rate in milliseconds. Must be greater than zero (default: `10`)
* `gcDurationBuckets`: Custom Prometheus buckets for Node.js GC duration histogram in seconds (default: `[0.001, 0.01, 0.1, 1, 2, 5]`)
* `core`: Kuzzle Core metrics directly extract from the `server:metrics` API action or from plugin inner logic.
* `monitorRequestDuration`: Enable/Disable request duration sampling (default: `true`)
* `prefix`: String to use to prefix metrics name (default: `kuzzle_`)
* `labels`: Additional labels to apply on all the different metrics (default: `{}`)
## Prometheus
### With only one Kuzzle node
```yaml
global:
scrape_interval: 10s # Set the scrape interval to every 10 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_configs:
- job_name: 'kuzzle'
metrics_path: /_metrics
params:
format: ['prometheus']
static_configs:
- targets: ['kuzzle:7512'] # the address of an application that exposes metrics for prometheus
```
### With an authentified user
If you use an other user than `anonymous` to expose the `server:metrics` API action, you will need to create a Kuzzle API Key (see [API Keys](https://docs.kuzzle.io/core/2/guides/advanced/api-keys/)) and use it to authentify the Prometheus scaper:
```yaml
global:
scrape_interval: 10s # Set the scrape interval to every 10 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_configs:
- job_name: 'kuzzle'
metrics_path: /_metrics
params:
format: ['prometheus']
authorization:
type: 'Bearer'
credentials: 'my-api-key'
static_configs:
- targets: ['kuzzle:7512'] # the address of an application that exposes metrics for prometheus
```
### With multiple Kuzzle nodes and using Docker Compose
If you use Docker Compose you'll need to provide the IP/Docker DNS name of each Kuzzle node as `targets`:
```yaml
global:
scrape_interval: 10s # Set the scrape interval to every 10 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
scrape_configs:
- job_name: 'kuzzle'
metrics_path: /_metrics
params:
format: ['prometheus']
static_configs:
- targets:
- 'kuzzle-plugin-prometheus-kuzzle-1:7512'
- 'kuzzle-plugin-prometheus-kuzzle-2:7512'
- 'kuzzle-plugin-prometheus-kuzzle-3:7512'
```
### Using Kubernetes annotations
If your Prometheus inside a Kubernetes cluster, you must use the helper HTTP route `/_/metrics` since Prometheus `params` configuration is not supported.
Your Pods annotations should look like this:
```yaml
metadata:
annotations:
prometheus.io/scrape: "true"
prometheus.io/path: /_/metrics
prometheus.io/port: "7512"
spec:
...
```
## Dashboards
### Features
You could find two dashboards in the `config/grafana/dashboards` folder:
- `kuzzle.json`: a dashboard with all the metrics exposed by the `server:metrics` API action with a `nodeId` filter and including:
- Active connections
- Active Realtime subscriptions
- Concurrent requests
- Pending requests
- Request per second
- Request duration
- Internal Errors
- `nodejs.json`: Node.js metrics dashboard with a `nodeId` filter and including:
- Process CPU Usage
- Process Memory Usage
- Process Restarts
- Event Loop Latency
- Heap Usage
You can import them both using the Grafana API, Web UI or the provisionning system (see the `docker-compose.yml` file).
### Screenshots
Kuzzle dashboard
Node.js dashboard
# Local development
You can run a local development stack using Docker Compose
```
$ docker-compose up
```
This will start a demonstration stack composed with:
* A Kuzzle server proxified by a Traefik router
* A Prometheus container configured to scrap metrics.
* A Grafana container.
Once started, go to `http://localhost:3000` and log in with the default Grafana credentials:
* username: `admin`
* password: `admin`
Make several requests using Kuzzle's HTTP API or SDKs, or by using the Admin Console.
> NOTE: You can also increase the number of Kuzzle nodes to test a cluster configuration.
> Use the Docker Compose `--scale` option to increase the number of replicas:
> ```
> docker-compose up -d --scale kuzzle=
> ```
> Notice that you need to update the `config/prometheus.yml` file to reflect the new number of nodes and restart the prometheus container using `docker restart `
# Migrations
## From version 3.x to 4.x
This new version 4.0.0 introduce numerous changes in the way metrics are collected and reported:
- The plugin now uses the `server:metrics` API action to retrieve metrics from Kuzzle Core. Calling the `server:metrics` API action with the `format` parameter set to `prometheus` will return metrics in the Prometheus format.
- The configuration of the plugin is now more flexible (see [Configuration](#configuration) section for more details):
- More control on the default Node.js metrics (set the Event Loop sample precision to custom value or adapt the Garbage Collector Prometheus bucket rates to fit your usecase).
- You can set different prefixes for Kuzzle Core metrics and Node.js metrics.
- The `nodeIP`, `nodeMAC` and `nodeHost` labels have beem removed in favor of the `nodeId` label.
- You can now disable the request recording job.
- Most of the metric names have been changed to be more consistent with Kuzzle Core metrics.
### Migration steps
Allow the user used by Prometheus to access server:metrics API action
Add the following rule to your user role to allow the user anonymous to access the server:metrics API action:
{
"controllers": {
// Your others controllers rules
"server": {
"actions": {
"metrics": true,
// ...
}
}
}
}
Update the plugin configuration
Here is the new default configuration file:
{
"plugins": {
"prometheus": {
"default": {
"enabled": true,
"prefix": "",
"eventLoopMonitoringPrecision": 10,
"gcDurationBuckets": [0.001, 0.01, 0.1, 1, 2, 5]
},
"core": {
"monitorRequestDuration": true,
"prefix": "kuzzle_"
}
}
}
}
Update your dashboards
If you have previously imported the example Grafana dashboard, you will have to update it to use the new metrics names or use the new ones located in config/grafana/dashboards.