Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ksmnetwork/polkadot-prometheus-logger
PolkaDOT Prometheus logger with Promtail, Loki and Grafana
https://github.com/ksmnetwork/polkadot-prometheus-logger
Last synced: about 2 months ago
JSON representation
PolkaDOT Prometheus logger with Promtail, Loki and Grafana
- Host: GitHub
- URL: https://github.com/ksmnetwork/polkadot-prometheus-logger
- Owner: ksmnetwork
- Created: 2022-02-05T15:09:20.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-02-06T11:42:53.000Z (over 2 years ago)
- Last Synced: 2024-04-16T06:46:50.834Z (6 months ago)
- Size: 12.7 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-dot - Polkadot Prometheus scrapper - PolkaDOT Prometheus logger with Promtail, (Libraries / Multisig Wallets)
README
Substrate/PolkaDOT Prometheus scrapper with curl and custom logging to the journaled system to receive the metrics in Grafana with Promntail including new or changed fields.
### A monitoring service on Grafana Cloud, Loki, Promtail, CURL ###
* No need to install Grafana
* No need to setup Loki
* Serveless Setup and Installation
* Full controll on what is send as logs
* Substrate changes can be obtained easy
* More....
---
## installation
* Create a bash script in "/usr/sbin/" and paste the code below, you can modify the scrapping time at ```sleep 15```
* Control what is logged to journald grep commnad at ```grep -v '#' | grep '_count'``` The current logged metric are all with ```_count```
* Change inside the script with your Substrate/Polkadio Prometheus PORT
* http://localhost:/metrics
* Or run polkadot daemon with a custom one by adding prometheus flags to your configuration file
* Specify Prometheus exporter TCP Port Flag --prometheus-port
```
#!/bin/bash
set -euf -o pipefail
exec 1> >(logger -s -t $(basename $0)) 2>&1
while true;
do
curl -sb -S http://localhost:/metrics | grep -v '#' | grep '_count'
sleep 15
done
```
* Create system unit to enable the scrip with "vi /etc/systemd/system/metrics.service"
```
[Unit]
Description=Prometheus Scrapper.
After=network.target
[Service]
Type=simple
ExecStart=/bin/bash /usr/sbin/metrics.sh
Restart=always
[Install]
WantedBy=multi-user.target
```
* Reload the systemctl daemon
```
systemctl daemon-reload
```
* Enable the service
```
systemctl enable metrics.service
```
* Start the service
```
systemctl start metrics.service
```
* Check the service
```
systemctl status metrics.service
```
---
## Promtail Installation ##
```
apt install sudo unzip -y \
&& sudo mkdir /opt/promtail \
&& cd /opt/promtail \
&& wget https://github.com/grafana/loki/releases/download/v2.2.1/promtail-linux-amd64.zip \
&& sudo unzip promtail-linux-amd64.zip
```
### Promtail Configuration template, change the `````` URL ###
###### URL Can be taken from Grafana Cloud Pannel ######
```
server:
http_listen_port: 0
grpc_listen_port: 0
log_level: error
clients:
- url:
positions:
filename: /tmp/positions.yaml
ignore_invalid_yaml: true
scrape_configs:
- job_name: journal
journal:
max_age: 60s
labels:
job: systemd-journal
relabel_configs:
- source_labels: ['__journal__systemd_unit']
target_label: 'unit'
- source_labels: ['__journal__hostname']
target_label: 'hostname'
- source_labels: ['__journal__host']
target_label: 'host'
pipeline_stages:
# Get this logs
- match:
selector: '{job="systemd-journal", unit="metrics.service"}'
stages:
- regex:
expression: '.*()'
- labels:
level:
output: log
stream: stream
timestamp: time
- timestamp:
source: timestamp
format: RFC3339Nano
- output:
source: output
```
* Test the config with promtail
```
./promtail-linux-amd64 -config.file ./substrate-metrics.yaml --dry-run
```
---
## Grafana Cloud ##
* Create Grafana Cloud Account and obtain your Loki URL with API key for Promtail
* Choose the label "UNIT" in your log query and look for "metrics.service" that is runing on the machine
* Query example:
```
{unit="metrics.service"} | regexp "(?P^[a-zA-Z0-9_]+)(?P\\{([^}]+)\\})\\W(?P[0-9]+)"
```
* Filtering on the queries with exact matching or regex
```
|= "polkadot_"
```
```
|= "substrate_"
```
```
|~ "substrate_.*kademlia"
```
* Example
```
{unit="metrics.service"} | regexp "(?P^[a-zA-Z0-9_]+)(?P\\{([^}]+)\\})\\W(?P[0-9]+)" |~ "substrate_.*kademlia"
```
### Dashboard download link, Granaba.com###
[Download link](https://grafana.com/grafana/dashboards/15674)
---
# For Support && Nominations #
* Display name for Validators. KSMNETWORK && KSMNETWORK-WEST
* Riot @gtoocool:matrix.org
* KUSAMA (KSM) Address
```H1bSKJxoxzxYRCdGQutVqFGeW7xU3AcN6vyEdZBU7Qb1rsZ```
* PolkaDOT (DOT) Address:
```15FxvBFDd3X7H9qcMGqsiuvFYEg4D3mBoTA2LQufreysTHKA```