https://github.com/myselfakashagarwal/monc

Containerised Three tier stack for monitoring servers including exposing, scrapping, storing, visualizing Logs & Metrics along with utilities built on it and Others !
https://github.com/myselfakashagarwal/monc

docker grafana loki monitoring node-exporter observability prometheus promtail

Last synced: about 2 months ago
JSON representation

Containerised Three tier stack for monitoring servers including exposing, scrapping, storing, visualizing Logs & Metrics along with utilities built on it and Others !

• Host: GitHub
• URL: https://github.com/myselfakashagarwal/monc
• Owner: myselfakashagarwal
• License: gpl-3.0
• Created: 2024-03-05T09:36:22.000Z (10 months ago)
• Default Branch: legacy
• Last Pushed: 2024-06-08T21:10:59.000Z (7 months ago)
• Last Synced: 2024-06-08T22:52:16.001Z (7 months ago)
• Topics: docker, grafana, loki, monitoring, node-exporter, observability, prometheus, promtail
• Homepage:
• Size: 23.4 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


## About

monc is a containerized stack for observability, which includes exposing, storing and visualization of logs and metrics. The stack is completely free from any dependencies and its abstracted to absolute simplification.

## Components 

- node-exporter - for exporting metrics

- promtail - for exporting logs 

- prometheus - for storing metrics

- loki - for storing logs 

- grafana - for visualizations 

### node-exporter 

The node exporter containerised setup mounts sys, proc and root file system on the container (in a secure manner). sys and proc for system generated metrics and root for disk monitoring. 

A process runs in the background which exposes all the collected metrics to {IP}:9100/metrics endpoint. matter fact the metrics generated by the container itself are flexiable but are ignored by default.

### promtail 

The promtail containerized setup mounts /var/log to container. A process runs in he background which pushed these logs to the loki instance, in a configured manner where jobs and other are defined.

### prometheus 

Prometheus scraps all the exposed metrics from exposed endpoints and store them in a time series data base. from this tsdb queries specificially PromQL are made to retrive data in specific order to visualize them (yes prometheus can visualize metrics but is not yet powerful as grafana)

### loki 

All the logs pushed by promtail are stored in the loki's chunks database from which they can be retrived via its own LogQL queries and visualize lateron.

### grafan 

Grafana is used for visualizations [the end], The data stored in loki and prometheus are treated as datasources thy which using queries dashboards can be built [they are beautiful]

### Working 

The stack is simplified being three tier it is modular in nature where user have flexibility to either setup only metrics monitoring or logs or both at the same time. matter fact the exporters both combined take about 40mb of memory and store services take about 100mb max combined.

with docker volumes are mounted which provide flexibility to backup and manage the data stored. 

### Usage 

The requirements for the setup is docker-compose thats it .. just clone the repo on the systems , entry the ip and config files paths [basic ones are included in the repo] in the .env file and spin the services.

The services are divided into profiles

`monc` → all services combined [tier1 + tier2 + tier3] 


`export` → export logs and metrics [tier1] 


`store` → store logs and metrics [tier2] 


`metrics` → setup export and store of metrics 
 

`logs` → setup export and store of logs 
 

`visualize` → setup visualizations [tier3] 


[assuming user is inside the repo and .env file is filled]

Start the stack

```bash

docker-compose -f monc.yml --profile monc --env-file=.env up

```