Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kubesphere/notification-manager
K8s native notification management with multi-tenancy support
https://github.com/kubesphere/notification-manager
alertmanager kubernetes
Last synced: 11 days ago
JSON representation
K8s native notification management with multi-tenancy support
- Host: GitHub
- URL: https://github.com/kubesphere/notification-manager
- Owner: kubesphere
- License: apache-2.0
- Created: 2020-03-24T16:06:57.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-04-03T07:20:01.000Z (8 months ago)
- Last Synced: 2024-04-13T21:09:43.444Z (7 months ago)
- Topics: alertmanager, kubernetes
- Language: Go
- Homepage:
- Size: 2.23 MB
- Stars: 211
- Watchers: 8
- Forks: 63
- Open Issues: 22
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Notification Manager
## Overview
Notification Manager manages notifications in multi-tenant K8s environment. It receives alerts, cloud event, and others (such as auditing, k8s events)
from different senders and then send notifications to various tenant receivers based on tenant label like `namespace` or `user`.
Supported senders includes:
- Prometheus Alertmanager
- Custom sender
- Prometheus (Coming soon)
- Cloud Event (Coming soon)
Supported receivers includes:
- [DingTalk](https://www.dingtalk.com/en)
- Email
- [Feishu](https://www.feishu.cn/en/)
- [Pushover](https://pushover.net/)
- SMS (Short Message Service)
- [Slack](https://slack.com/)
- Webhook
- [WeChat](https://work.weixin.qq.com/)
- [Discord](https://discord.com/)
## Architecture
Notification Manager uses [Receiver](docs/crds/receiver.md) and [Config](docs/crds/config.md) CRDs to store notification configs
like email, WeChat and slack. It also includes an operator to create and reconcile [NotificationManager](docs/crds/notification-manager.md)
CRD which watches all [receivers](docs/crds/receiver.md) and [configs](docs/crds/config.md), updates notification settings accordingly and sends notifications to users.
## Process
The incoming data (alert, cloud event and others) will cache in the cache firstly, then goes through steps such as [silence](#silence), inhibit (coming soon), [route](#route),
[filter](#filter), [aggregation](#aggregation), etc. Notifications will generate from data using [template](#customize-template),
then send to receivers and [history webhook](#history) (if set).
### Silence
`Silence` is a straightforward way to simply mute notifications for a given time. It uses [Silence](docs/crds/silence.md) CRD to define
the silence policy. If incoming data matches an active silence, no notifications will be sent out for that data.
### Route
`Route` find all receivers the notifications will send to.
There are two ways to determine which receivers the notifications will send to, one is via [Router](docs/crds/router.md) CRD,
and the other is auto-matching via the `namespace` label in the notification.
Usually the incoming data contains a `namespace` label, Notification Manager uses this label to decide which receiver to use for sending notifications:
- For KubeSphere, Notification Manager will try to find tenants with the right to access the namespace from [sidecar](docs/crds/notification-manager.md#tenant-sidecar)
and then find receivers with `user = xxx` label.
- For Kubernetes, Notification Manager will try to find receivers with `namespace = xxx` label.
For data without a `namespace` label, for example alerts of node or kubelet, user can set up a receiver with `type = global` label to receive notifications without a `namespace` label. A global receiver sends notifications for all notifications received regardless any label. A global receiver usually set for an admin role.
How the two methods work together is determined by the [routePolicy](docs/crds/notification-manager.md#RoutePolicy).
### Filter
`Filter` filters the notifications sent to receivers. There are two ways to filter notifications. One is using [alertSelector](docs/crds/receiver.md#notification-filter) in the receiver,
the other is using [tenant silence](docs/crds/silence.md).
### Aggregation
`Aggregation` groups notifications by [groupLabels](docs/crds/notification-manager.md#grouplabels). Notifications in the same group will send together.
### History
`History` is a webhook used to collect all notifications sent to receivers, it can be set via [history](docs/crds/notification-manager.md#history).
## QuickStart
### Install
We assume you already have a Kubernetes cluster (v1.16+). You can install one using [KubeKey](https://github.com/kubesphere/kubekey) if you haven't.
#### Install with yaml
```shell
# Deploy CRDs and the Notification Manager Operator:
kubectl apply -f https://github.com/kubesphere/notification-manager/releases/download/v2.5.2/bundle.yaml
# Deploy default template:
kubectl apply -f https://github.com/kubesphere/notification-manager/releases/download/v2.5.2/template.yaml
# Deploy built-in language packs.
kubectl apply -f https://github.com/kubesphere/notification-manager/releases/download/v2.5.2/zh-cn.yaml
```
#### Install with helm
```shell
helm install notification-manager --create-namespace -n kubesphere-monitoring-system https://github.com/kubesphere/notification-manager/releases/download/v2.5.2/notification-manager.tgz
```
### Configure NotificationManager
[NotificationManager](docs/crds/notification-manager.md) CRD Defines the desired notification manager deployment. The Notification Manager Operator
ensures a deployment meeting the resource requirements is running.
We should create a NotificationManager CR first, skip this when using helm install.
```shell
kubectl apply -f https://github.com/kubesphere/notification-manager/releases/download/v2.5.2/notification_manager.yaml
```
### Configure sender
Notification Manager uses port `19093` and API path `/api/v2/alerts` to receive alerts sent from Alertmanager.
#### Config Alertmanager to send alerts to Notification Manager
To receive Alertmanager alerts, add webhook config like below to the `receivers` section of Alertmanager configuration file:
```shell
"receivers":
- "name": "notification-manager"
"webhook_configs":
- "url": "http://notification-manager-svc.kubesphere-monitoring-system.svc:19093/api/v2/alerts"
```
#### Customize sender
Below is the data structure passed to the notification manager, please refer to [Data](https://github.com/kubesphere/notification-manager/blob/master/pkg/template/types.go#L12) for more details.
```yaml
{
"alerts": [
{
"status": "firing",
"labels": {
"alertname": "KubePodCrashLooping",
"container": "busybox-3jb7u6",
"instance": "10.233.71.230:8080",
"job": "kube-state-metrics",
"namespace": "pp1",
"pod": "dd1-0",
"prometheus": "kubesphere-monitoring-system/k8s",
"severity": "critical"
},
"annotations": {
"message": "Pod pp1/dd1-0 (busybox-3jb7u6) is restarting 1.07 times / 5 minutes.",
},
"startsAt": "2020-02-26T07:05:04.989876849Z",
"endsAt": "0001-01-01T00:00:00Z",
}
],
}
```
A custom senders can send notifications using notification manager simply by sending data to `http://notification-manager-svc.kubesphere-monitoring-system.svc:19093/api/v2/alerts`.
### Create receiver and config
Now it's time to create the receiver and config to receive notifications, you can find guides to create them in [receiver](docs/crds/receiver.md) and [config](docs/crds/config.md).
### Customize template
To customize the notification template, please refer to [template](docs/template.md).
## Development
```
# Build notification-manager-operator and notification-manager docker images
make build
# Push built docker images to docker registry
make push
```
## Documentation
- [API documentation](./docs/api/_index.md).