https://github.com/oxide-byte/rust-observability

https://github.com/oxide-byte/rust-observability

Last synced: 9 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/oxide-byte/rust-observability
• Owner: oxide-byte
• License: mit
• Created: 2025-08-22T19:42:39.000Z (10 months ago)
• Default Branch: master
• Last Pushed: 2025-08-22T20:57:24.000Z (10 months ago)
• Last Synced: 2025-08-22T22:52:53.033Z (10 months ago)
• Language: Rust
• Size: 20.5 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Observability

## Introduction

A service that is not observed, is lost in the universe of distributed systems. This POC-project is a small sample to

show how keep track on your Rust Service. The Service and all the infrastructure is dockerized, you can simply start up

all with "docker-compose up"

The project demonstrates two methods of log collection:

1. Via `tracing-loki` directly from the application.

2. Via Promtail scraping Docker logs.

It also showcases a robust metrics collection system, capturing both application-level metrics (HTTP requests, handler

durations) and process-level metrics (CPU, memory).

The project uses a modern stack (Axum, Tokio, OpenTelemetry) and integrates the standard observability tools (

Prometheus, Grafana, Loki)

A small Screenshot the observed service in Grafana:

![alt text](doc/images/sample.png "sample")

Logs:

![alt text](doc/images/log.png "log")

## Links

The current links available by the application:

The Rust Service:

* Hosted on port 8080: http://127.0.0.1:8080/

* API: http://127.0.0.1:8080/api/demo for generating log entries

* Metrics: http://127.0.0.1:8080/metrics for collecting the current metrics of the Rust Service

* Health: http://127.0.0.1:8080/metrics for healths status

Prometheus:

* http://127.0.0.1:9090/

Grafana:

* http://127.0.0.1:3000/

Loki:

* A simple query on Loki: http://127.0.0.1:3100/loki/api/v1/query?query={job="docker-logs"}

* A query with a range option:

    - http://127.0.0.1:3100/loki/api/v1/query_range?query={job="docker-logs"}&limit=1000&direction=backward&start=<

      rfc3339_or_unix_ns>&end=

## Architecture: Logs and Metrics Flow

A little Flowchart for illustrating how the different logs are collected and transmitted to the different systems:

```mermaid

flowchart LR

  subgraph Docker Network

    A[Rust-Axum-Server]

    D[(Docker JSON logs
/var/lib/docker/containers/*/*-json.log)]

    PT[Promtail]

    L[Loki]

    PR[Prometheus]

    G[Grafana]

  end

  %% Logs flow (1)

  A -- stdout logs --> D

  D -- tail --> PT

  PT -- push /loki/api/v1/push --> L

  G -- query (Loki datasource) --> L

  %% Logs flow (2)

  A -- push (tracing-loki) --> L

  %% Metrics flow

  A -- scrape /metrics --> PR

  PT -- scrape /metrics (9080) --> PR

  L -- scrape /metrics (3100) --> PR

  G -- query (Prometheus datasource) --> PR

```

Notes:

- Promtail collects logs from Docker JSON log files and pushes them to Loki. Loki does not pull logs from the app. (

  Solution 1)

- Cargo tracing-loki, declare appender, push to Loki (Solution 2)

- Prometheus scrapes metrics from the app, Promtail, and Loki; logs are not stored in Prometheus.

- Grafana uses the Loki datasource for logs and the Prometheus datasource for metrics.

## Docker-Compose

Starting the application and all infrastructure components with:

```shell

docker-compose up

```