{"id":19396071,"url":"https://github.com/ruanbekker/docker-loki-distributed-minio","last_synced_at":"2025-04-24T05:30:52.256Z","repository":{"id":139706831,"uuid":"301404056","full_name":"ruanbekker/docker-loki-distributed-minio","owner":"ruanbekker","description":"Distributed Loki setup with BoltDB and Minio using Docker","archived":false,"fork":false,"pushed_at":"2022-04-07T08:30:53.000Z","size":46,"stargazers_count":10,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-17T02:14:41.822Z","etag":null,"topics":["boltdb","docker","grafana","loki","minio"],"latest_commit_sha":null,"homepage":"https://ruan.dev","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ruanbekker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-05T12:38:11.000Z","updated_at":"2024-09-03T07:28:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"ce39f2de-80f4-4567-b14b-c9638da37053","html_url":"https://github.com/ruanbekker/docker-loki-distributed-minio","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruanbekker%2Fdocker-loki-distributed-minio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruanbekker%2Fdocker-loki-distributed-minio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruanbekker%2Fdocker-loki-distributed-minio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruanbekker%2Fdocker-loki-distributed-minio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ruanbekker","download_url":"https://codeload.github.com/ruanbekker/docker-loki-distributed-minio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250572153,"owners_count":21452325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boltdb","docker","grafana","loki","minio"],"created_at":"2024-11-10T10:33:24.788Z","updated_at":"2025-04-24T05:30:52.244Z","avatar_url":"https://github.com/ruanbekker.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# docker-loki-distributed-minio\nDistributed Loki setup with BoltDB and Minio using Docker\n\n## About\n\nThere's options for using different ring backend storage:\n\n- The `docker-compose.yml` provides a distributed loki stack with **etcd** as the backend kvstore\n- The `docker-compose-consul.yml` provides a distrubuted loki stack with **consul** as the backend kvstore\n- The `docker-compose-traefik.yml` provides a distrubuted loki stack with **etcd** as the backend kvstore\n\n## Components\n\nThis stack consists of:\n\n- loki-distributor (promtail or docker logging driver points to this component)\n- loki-querier (grafana or logcli points to this component)\n- loki-ingester\n- loki-table-manager\n- redis\n- etcd or consul\n\nThe architecture document can be viewed [here](https://grafana.com/docs/loki/latest/fundamentals/architecture/)\n\n## Pre-Requisites\n\nFor your containers to use to loki logging driver, you need to install it using:\n\n```bash\n$ docker plugin install grafana/loki-docker-driver:latest --alias loki --grant-all-permissions\n```\n\n## Logging Options\n\nTo use loki logging driver (push logs directly to loki):\n\n```yaml\n    logging:\n      driver: loki\n      options:\n        loki-url: http://localhost:3101/loki/api/v1/push\n        loki-external-labels: job=dockerlogs,environment=development\n```\n\nTo use json file logging driver (and let promtail scrape the logs):\n\n```yaml\n    logging:\n      driver: \"json-file\"\n      options:\n        max-size: \"1m\"\n        max-file: \"1\"\n        tag: \"{{.Name}}\"\n```\n\n## Boot the Stack:\n\nI will boot the stack that includes consul as a backend:\n\n```bash\ndocker-compose -f docker-compose-consul.yml up -d --build\n```\n\n## Make HTTP Requests\n\nTest the flask-app:\n\n```bash\ncurl 'http://localhost:5000/?msg=hi'\n```\n\nTest the nginx-app:\n\n```bash\ncurl 'http://localhost:8084/?msg=bye'\n```\n\n## Example Queries\n\nQuery labels as k/v pairs using `logfmt`:\n\n```\n{container_name=\"flask-app\"} | logfmt |= \"scheme=http\"\n```\n\nMetric Queries: Log Entries sum by `container_name`:\n\n```\nsum(rate({job=\"containerlogs\"}[30s])) by (container_name)\n```\n\n## Resources\n\nExtra credit goes to the ones in the below issues, as I retrieved most of the loki config from there:\n\n- https://github.com/grafana/loki/issues/2155\n- https://github.com/grafana/loki/issues/1434\n\nDocumentation on Loki:\n\n- https://grafana.com/docs/loki/latest/configuration/examples/#almost-zero-dependencies-setup\n- https://grafana.com/docs/loki/latest/operations/storage/boltdb-shipper/\n- https://grafana.com/blog/2020/10/28/loki-2.0-released-transform-logs-as-youre-querying-them-and-set-up-alerts-within-loki/\n\nExtra resources to add a Go app to this stack:\n\n- https://medium.com/martinomburajr/building-a-go-web-app-from-scratch-to-deploying-on-google-cloud-part-4a-containerizing-our-go-e1ae2b152ee2\n- https://semaphoreci.com/community/tutorials/how-to-deploy-a-go-web-application-with-docker\n- https://github.com/shijuvar/golang-docker/blob/master/main.go\n\nSimilar:\n\n- https://github.com/CloudXiaobai/loki-cluster-deploy/tree/master/demo/docker-compose and [blog](https://www.mdeditor.tw/pl/pFBb/zh-hk)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruanbekker%2Fdocker-loki-distributed-minio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruanbekker%2Fdocker-loki-distributed-minio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruanbekker%2Fdocker-loki-distributed-minio/lists"}