{"id":18439684,"url":"https://github.com/matjaz99/alertmonitor","last_synced_at":"2025-04-07T21:32:47.900Z","repository":{"id":42644330,"uuid":"187508808","full_name":"matjaz99/alertmonitor","owner":"matjaz99","description":"Display Prometheus alerts","archived":false,"fork":false,"pushed_at":"2024-09-15T22:55:16.000Z","size":5712,"stargazers_count":6,"open_issues_count":9,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-23T01:03:23.051Z","etag":null,"topics":["alarms","alertmanager","alertmonitor","alerts","correlation","docker","metrics","monitoring","prometheus","webapp","webhook"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matjaz99.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-19T17:43:06.000Z","updated_at":"2024-09-15T22:55:19.000Z","dependencies_parsed_at":"2024-02-10T23:28:48.279Z","dependency_job_id":"8e538088-efcc-4be1-b426-a5867daeac6d","html_url":"https://github.com/matjaz99/alertmonitor","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matjaz99%2Falertmonitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matjaz99%2Falertmonitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matjaz99%2Falertmonitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matjaz99%2Falertmonitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matjaz99","download_url":"https://codeload.github.com/matjaz99/alertmonitor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247732756,"owners_count":20986913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alarms","alertmanager","alertmonitor","alerts","correlation","docker","metrics","monitoring","prometheus","webapp","webhook"],"created_at":"2024-11-06T06:26:07.363Z","updated_at":"2025-04-07T21:32:42.888Z","avatar_url":"https://github.com/matjaz99.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Alertmonitor for Prometheus\n\n![GitHub Project](https://img.shields.io/badge/app-alertmonitor-crimson)\n![GitHub release](https://img.shields.io/github/v/release/matjaz99/alertmonitor)\n[![Docker Pulls](https://img.shields.io/docker/pulls/matjaz99/alertmonitor.svg)](https://hub.docker.com/r/matjaz99/alertmonitor)\n[![GitHub issues](https://img.shields.io/github/issues/matjaz99/alertmonitor.svg)](https://GitHub.com/matjaz99/alertmonitor/issues/)\n\nAlertmonitor is a webapp for displaying alerts from Prometheus or other data sources. \nAlerts are received from Alertmanager (push notifications) or they can be retrieved directly from Prometheus (polling).\n\nCombination of both push and poll mechanisms assure always up-to-date state of alerts without any data loss. \n\nAlertmonitor receives alerts from Alertmanager on the HTTP endpoint: `/alertmonitor/webhook`. \nAlternatively, if webhook receiver is not configured, Alertmonitor can pull alerts directly from Prometheus. \nIdeally both approaches should be used in combination. This way you'll always receive alert immediately when it is fired, and yet it \n*synchronizes* alerts with Prometheus in case if any alert (or clear) has been lost.\n\nAlertmonitor automatically correlates firing and resolving alerts to display current state of active alarms.\n\nAlertmonitor displays monitored objects as Instances or SmartTargets. Instance is one particular exporter on a server, while \nSmartTarget combines all instances on the same server and displays all active alerts on that server.\n\nAlertmonitor supports PromQL for making queries and range queries.\n\nTags provide a quick way of filtering alerts.\n\nAlertmonitor can store alerts in MongoDB database. It stores all incoming messages \nand whole journal (history) of alerts.\n\nScreenshots:\n\n![Alertmonitor](docs/screenshots/2.0.0/alerts.png) \n\n![Alertmonitor](docs/screenshots/2.0.0/targets.png) \n\n![Alertmonitor](docs/screenshots/2.2.0/query.png)\n\n\n## Quick start\n\nDeploy Alertmonitor on Docker:\n\n```\ndocker run -d -p 8080:8080 matjaz99/alertmonitor:latest\n```\n\nOpen web browser and go to address: [http://hostname:8080/alertmonitor/](http://hostname:8080/alertmonitor/)\n\n\nThere is also `docker-compose.yml` file available for deployment in Swarm cluster.\n\n\n### Docker images\n\nDocker images are available on Docker hub: [https://hub.docker.com/r/matjaz99/alertmonitor](https://hub.docker.com/r/matjaz99/alertmonitor)\n\n\n## Configure alerts in Prometheus\n\nAlertmonitor strongly relies on properly configured labels in alert rules. \nPlacing additional labels into alert rules will enrich the information that alert carries, such as: \nseverity, metric labels, current metric value, alert tags or group name.\n\n### Labeling alerts\n\nAlertmonitor recognizes the following labels:\n\n| Label | Description |\n|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| severity | Optional (default=indeterminate). Severity is the weight of event. Possible values: `critical`, `major`, `minor`, `warning`, `clear` and `informational` |\n| priority | Optional (default=low). Priority tells how urgent is alarm. Possible values: `high`, `medium`, `low` |\n| info | Mandatory. Detailed information about the alert. **Important: Info may not contain variables which change over the time (such as current metric value), because it creates new time series of alerts each time and the correlation will not work.!** |\n| instance | Optional. `instance` is usually already included in metric, but sometimes if alert rule doesn't return instance, you can provide its value here by any other means. Usually IP address and port of exporter. |\n| nodename | Optional. Name of this instance. |\n| tags | Optional. Custom tags that describe the alert (comma separated). Tags are used for quick filtering in Alertmonitor. |\n| group | Optional. Custom group name. |\n| url | Optional. Custom URL that is related to alert. |\n| eventType | Optional. Event type according to IUT-T X.733 recommendation |\n| probableCause | Optional. Probable cause according to IUT-T X.733 recommendation |\n| description | Optional. Additional description. Value is read from a labels section if it exists, otherwise from annotations section. |\n| currentValue | Optional. Current metric value. Get it with: `{{ humanize $value }}`. Append units (eg. % or MB) if you need to do so. **Important: Current value may not be in `labels` section of alert rule but inside `annotations`!** |\n\n\u003e Alert's `correlationId` is defined by: `alertname`, `info`, `instance` and `job`. Clear event should produce the same `correlationId`.\n\n\u003e It is highly recommended to write alert rules in such manner that `instance`, `job` and `info` are always present in the resulting alert.\n\nExample of alert rule in Prometheus:\n\n```yaml\ngroups:\n- name: my-alerts\n rules:\n - alert: High CPU usage\n expr: sum(rate(process_cpu_seconds_total[5m])) by (instance) * 100 \u003e 80\n for: 1m\n labels:\n severity: critical\n info: CPU alert for Node '{{ $labels.instance }}'\n tags: hardware, server, cpu, overload\n url: 'http://${GRAFANA_HOSTNAME}/dashboard/'\n description: Node {{ $labels.instance }}\n annotations:\n currentValue: '{{ $value }}%'\n```\n\n\n### Configure webhook receiver in Alertmanager\n\nIn order to receive alerts, configure an Alertmonitor receiver endpoint in `alertmanager.yml` configuration file.\n\n```yaml\nroute:\n receiver: alertmonitor\n group_by: [alertname]\n group_wait: 10s\n group_interval: 5m\n repeat_interval: 3h\n\nreceivers:\n- name: alertmonitor\n webhook_configs:\n - url: http://alertmonitor:8080/alertmonitor/webhook\n send_resolved: true\n```\n\n## Data Providers\n\nSince version 3, Alertmonitor supports Data Providers. Data providers are sources that will send data to \nAlertmonitor. It is possible to define more Prometheus servers or some other sources.\n\nAlertmonitor supports the following data providers:\n- Prometheus\n- Eventlogger\n\nData Providers are configured in `providers.yml` configuration file.\n\nIf no providers are defined, there still exists a single data provider (called `.default`) that is constructed from \nthe environment variables and is bound to the `/alertmonitor/webhook` URI endpoint. All requests received on that \nendpoint will be processed by `.default` data provider process.\n\n\u003e All URI endpoints **must** start with `/alertmonitor/webhook`.\n\n\u003e It is not recommended to configure two or more providers to send events to the same URI endpoint!\n\n\n## Alertmonitor GUI\n\nAlertmonitor offers the following views.\n\n### Active alerts\n\nThis view shows all currently active alerts.\n\nActive alerts can be filtered by selecting one or more tags. Deselect all tags to show all alerts.\n\nSearch (in upper right corner) allows searching alerts by: `instance`, `alertname`, `info`, `job`, `description`. Search can \nbe used in combination with tags.\n\n### Query\n\nQuery view provides a GUI for executing PromQL queries. Two types of queries are supported: `query` and `query_range`. \nWhen executing query range, start/end date and step are configurable.\n\n### Journal\n\nThis view shows all history of received events. The size of journal is limited by `ALERTMONITOR_JOURNAL_SIZE` parameter. \nWhen journal reaches its maximum size, the oldest events will be removed (First in, first out).\n\nRemark: all the data in Alertmonitor is based on journal events. For example, Alertmonitor can only show targets, \nwhich have at least one alert recorded in journal.\n\n### Webhook\n\nThis view shows raw messages as they were received by the HTTP webhook.\n\n### Target\n\nAlertmonitor strips protocol and port from `instance` label and what remains is target's hostname or IP address or FQDN.\n\nAlertmonitor then filters alerts and displays those who's hostnames match.\n\nEach target shows number of active alerts and an icon indicating the highest severity of raised alert.\n\n### Providers\n\nShow configured data providers.\n\nQuick statistics is shown for each data provider:\n- number of alerts by severity\n- number of received messages/alerts\n- timers (up time, time since last event...)\n- synchronization success rate\n\n### Report\n\nReport is an overview of the whole monitoring system. It shows some statistics of Alertmonitor \nand Prometheus. It also shows the status of instances, average availability and other KPIs. \nMost KPIs and metrics are queried from Prometheus.\n\n### Configuration\n\nHere it is possible to change some configuration parameters during runtime.\n\n### About\n\nApplication meta data, version, build info, maintainers, public channels...\n\n\n## Persistence\n\nAlertmonitor supports the following data storage options:\n- In memory\n- MongoDB\n- Kafka (experimental)\n\nIn memory option works out-of-the-box without any configuration.\n\nTo enable MongoDB data storage, set the environment variable `ALERTMONITOR_MONGODB_ENABLED` to `true` \nand configure `ALERTMONITOR_MONGODB_CONNECTION_STRING`. MongoDB is disabled by default. \nData is deleted according to retention policy (`ALERTMONITOR_DATA_RETENTION_DAYS`). \nAlertmonitor will automatically create new database `alertmonitor` in MongoDB, if it doesn't exist yet.\n\nKafka is experimental feature. If enabled, Alertmonitor will send all alerts to Kafka topic. \nThis feature might be useful for further processing and analytics of alarms.\n\n\n## Configuration\n\n### Application configuration\n\nAlertmonitor can be configured with environment variables. Variables starting with `ALERTMONITOR_*` are related \nto behaviour of the application, while other variables may be used for other purposes such as log \nconfiguration or custom environment variable substitution.\n\n\u003e With environment variables, only `.default` data provider (Prometheus) can be configured. Other providers \nare configured in `providers.yml` (see Data Providers). If providers config file is not found, default data provider \nis constructed from environment variables.\n\nA list of supported environment variables:\n\n| EnvVar | Description |\n|--------------------------------------------------|--------------------------------------------------------------------------------------------------------|\n| ALERTMONITOR_DATAPROVIDERS_CONFIG_FILE | File path to providers.yml configuration file. Default: /opt/alertmonitor/providers.yml |\n| ALERTMONITOR_DATAPROVIDERS_DEFAULT_PROVIDER_NAME | Name of default data provider. Default: .default |\n| ALERTMONITOR_DATA_RETENTION_DAYS | History data in days. Default: 7 |\n| ALERTMONITOR_SYNC_INTERVAL_SEC | Periodic synchronisation interval in seconds. Default: 900 |\n| ALERTMONITOR_PROMETHEUS_SERVER | The URL of Prometheus server. Default: http://localhost:9090 |\n| ALERTMONITOR_PROMETHEUS_CLIENT_POOL_SIZE | Pool size of http clients for communication with Prometheus API. Default: 1 |\n| ALERTMONITOR_HTTP_CLIENT_READ_TIMEOUT_SEC | Timeout of http client requests. Default: 120 |\n| ALERTMONITOR_HTTP_CLIENT_CONNECT_TIMEOUT_SEC | Connection timeout of http client. Default: 10 |\n| ALERTMONITOR_DATE_FORMAT | Date format for displaying in GUI. Default: yyyy/MM/dd H:mm:ss |\n| ALERTMONITOR_KAFKA_ENABLED | Enable or disable publishing to Kafka. This is experimental feature! Default: false |\n| ALERTMONITOR_KAFKA_SERVER | Hostname and port for Kafka. Default: hostname:9092 |\n| ALERTMONITOR_KAFKA_TOPIC | Name of topic. Default: alertmonitor_notifications |\n| ALERTMONITOR_MONGODB_ENABLED | Enable or disable storing data to MongoDB. If disabled, data is stored in memory only. Default: false | \n| ALERTMONITOR_MONGODB_CONNECTION_STRING | The connection string for MongoDB (username, password and host). |\n\n\n\n### Security\n\n#### Secure HTTPS protocol\n\nAlertmonitor supports `http` and `https` protocols for communication with data providers.\n\nIn case of `https`, no certificate validation is checked.\n\n#### Basic authentication\n\nHttp client in Alertmonitor supports basic authentication when connecting to the data provider (only for https clients).\n\nCurrently, username and password can only be provided via `ALERTMONITOR_PROMETHEUS_SERVER` variable using syntax `https://username:password@hostname:port`.\n\n\n### Environment variable substitution\n\nPrometheus doesn't support substitution of environment variables in alert rules. Alertmonitor does that for you.\n\nEnvironment variables can be set on system level or directly on docker containers. Example in docker-compose file.\n\n```yaml\n environment:\n - GRAFANA_HOSTNAME: my.grafana.domain\n```\n\nTemplate in labels to be replaced: `${GRAFANA_HOSTNAME}`.\n\nAlertmonitor will automatically replace all occurrences of templates with corresponding environment variables.\n\nExample when comes this handy: you may link an alert with Grafana dashboard by using `url` label: `http://${GRAFANA_HOSTNAME}/dashboard`. \nAlertmonitor will replace all occurrences of template with corresponding environment variable: \n`url: http://my.grafana.domain/dashboard`.\n\nEnvironment variable substitution works on the following labels:\n- `nodename`\n- `info`\n- `tags`\n- `url`\n- `description`\n\n\n## Metrics\n\nMetrics are available in Prometheus format on URI endpoint:\n\n```\nGET /alertmonitor/metrics\n```\n\nAlertmonitor supports the following metrics in Prometheus format:\n- `alertmonitor_build_info`\n- `alertmonitor_providers_info`\n- `alertmonitor_webhook_requests_received_total`\n- `alertmonitor_journal_events_total`\n- `alertmonitor_active_alerts_count`\n- `alertmonitor_alerts_balance_factor`\n- `alertmonitor_last_event_timestamp`\n- `alertmonitor_prom_api_duration_seconds`\n- `alertmonitor_sync_success`\n- `alertmonitor_sync_interval_seconds`\n- `alertmonitor_db_inserts_total`\n- `alertmonitor_db_queries_total`\n- `alertmonitor_db_updates_total`\n- `alertmonitor_db_deletes_total`\n- `alertmonitor_db_failures_total`\n- `alertmonitor_memory_total_bytes`\n- `alertmonitor_memory_free_bytes`\n- `alertmonitor_memory_max_bytes`\n- `alertmonitor_available_processors`\n\n\n## Log files\n\nInside container log files are located in directory `/opt/alertmonitor/log`.\n\nConfigure the log file location with environment variable `SIMPLELOGGER_FILENAME=/opt/alertmonitor/log/alertmonitor.log`\n\nRolling file policy can be also configured. For complete configuration of simple-logger visit [https://github.com/matjaz99/simple-logger](https://github.com/matjaz99/simple-logger)\n\n\n## For developers\n\n### Community\n\nGoogle group for Alertmonitor Users:\n\nhttps://groups.google.com/g/alertmonitor-users\n\n### Dependencies\n\nAlertmonitor is written in Java. It's a maven project. It runs as web app on Apache Tomcat server and uses JSF 2.2 with Primefaces 12.0 for frontend interface.\nIn version 1.5.1 I switched from Java 8 to Java 13. I had to add `javax.annotations` dependency to pom.xml file.\n\n### Simple-logger maven dependency\n\nSorry, simple-logger is not available on Maven central repo. You can either build it on your own \nor download jar file from [here](http://matjazcerkvenik.si/download/simple-logger-1.7.0.jar). \nThen manually import it into your local repository:\n\n```\nwget http://matjazcerkvenik.si/download/simple-logger-1.7.0.jar\n\nmvn install:install-file -Dfile=simple-logger-1.7.0.jar -DgroupId=si.matjazcerkvenik.simplelogger -DartifactId=simple-logger -Dversion=1.7.0 -Dpackaging=jar\n```\n\nRun the project with maven:\n\n```\nmvn tomcat7:run\n```\n\n### Docker\n\nBuild docker image and push to docker hub:\n\n```\ndocker build -t {{namespace}}/{{image}}:{{tag}} .\ndocker push {{namespace}}/{{image}}:{{tag}}\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatjaz99%2Falertmonitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatjaz99%2Falertmonitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatjaz99%2Falertmonitor/lists"}