{"id":28504225,"url":"https://github.com/function61/promswarmconnect","last_synced_at":"2025-07-06T04:31:17.556Z","repository":{"id":57512431,"uuid":"77714518","full_name":"function61/promswarmconnect","owner":"function61","description":"Bridges Docker Swarm services to Prometheus without any changes to Prometheus","archived":false,"fork":false,"pushed_at":"2024-02-24T10:13:47.000Z","size":163,"stargazers_count":24,"open_issues_count":1,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-08T18:03:51.930Z","etag":null,"topics":["docker","prometheus","service-discovery","swarm-service"],"latest_commit_sha":null,"homepage":"https://function61.com/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/function61.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/security.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-30T22:52:19.000Z","updated_at":"2024-02-24T10:13:51.000Z","dependencies_parsed_at":"2024-06-20T05:59:43.138Z","dependency_job_id":null,"html_url":"https://github.com/function61/promswarmconnect","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/function61/promswarmconnect","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/function61%2Fpromswarmconnect","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/function61%2Fpromswarmconnect/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/function61%2Fpromswarmconnect/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/function61%2Fpromswarmconnect/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/function61","download_url":"https://codeload.github.com/function61/promswarmconnect/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/function61%2Fpromswarmconnect/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263848846,"owners_count":23519525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","prometheus","service-discovery","swarm-service"],"created_at":"2025-06-08T18:03:35.880Z","updated_at":"2025-07-06T04:31:17.537Z","avatar_url":"https://github.com/function61.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Build status](https://github.com/function61/promswarmconnect/workflows/Build/badge.svg)\n[![Download](https://img.shields.io/docker/pulls/fn61/promswarmconnect.svg?style=for-the-badge)](https://hub.docker.com/r/fn61/promswarmconnect/)\n\n\n\u003e [!WARNING]  \n\u003e [Prometheus recently added native Swarm support](https://prometheus.io/docs/guides/dockerswarm/) (therefore this project might be considered unnecessary)\n\u003e \n\u003e For more details see [Issue #14](https://github.com/function61/promswarmconnect/issues/14)\n\n\nWhat?\n-----\n\nSyncs services/tasks from Docker Swarm to Prometheus by pretending to be a Triton service\ndiscovery endpoint, which is a\n[built-in service discovery module](https://github.com/prometheus/prometheus/tree/master/discovery/triton)\nin Prometheus.\n\nFeatures:\n\n- Have your container metrics scraped fully automatically to Prometheus.\n- We don't have to make ANY changes to Prometheus (or its container) to support Docker\n  Swarm (except configuration changes).\n- Supports overriding metrics endpoint (default `/metrics`) and port.\n- Supports clustering, so containers are discovered from all nodes. Neither Prometheus\n  nor promswarmconnect needs to run on the Swarm manager node.\n    * promswarmconnect needs to run on Swarm manager if you use the `docker.sock` mount option\n- Supports scoping Prometheus `job` label to a) container (default), b) host (think host-level\n  metrics) or c) static string (think cluster-wide metrics).\n  [Read more](https://github.com/function61/promswarmconnect/blob/ecc947d4aa6b29bb4595929d2bc23b1ec7bd5e9e/cmd/promswarmconnect/main.go#L173)\n\n![](docs/architecture.png)\n\nNOTE: the drawing is for option 2). This is even simpler if you use option 1) with socket mount.\n\n\nHow to deploy\n-------------\n\nRun the image from Docker Hub (see top of README) with the configuration mentioned below.\nBoth options mention \"VERSION\" version of the image. You'll find the latest version from\nthe Docker Hub. We don't currently publish \"latest\" tag so the versions are immutable.\n\nYou need to run promswarmconnect and Prometheus on the same network.\n\n### Option 1: run on Swarm manager node with mounted `docker.sock`\n\nThis is the easiest option, but requires you to have a placement constraint to guarantee\nthat promswarmconnect always runs on the manager node - its Docker socket is the only API\nwith knowledge of the whole cluster state.\n\n```console\n$ docker service create \\\n\t--name promswarmconnect \\\n\t--constraint node.role==manager \\\n\t--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock \\\n\t--env \"DOCKER_URL=unix:///var/run/docker.sock\" \\\n\t--env \"NETWORK_NAME=yourNetwork\" \\\n\t--network yourNetwork \\\n\t\"fn61/promswarmconnect:VERSION\"\n```\n\nNOTE: `unix:..` contains three forward slashes!\n\n\n### Option 2: run on any node by having Docker's socket exposed over HTTPS\n\nThis may be useful to you if you have other needs that also require you to expose Docker's\nport. For example I'm running [Portainer](https://www.portainer.io/) on my own computer\nand that needs to dial to Docker's socket over TLS from the outside world.\n\nDocker's socket needs to be exposed over HTTPS with a client cert authentication. We use\n[dockersockproxy](https://github.com/function61/dockersockproxy) for this. You can do the\nsame with just pure Docker (expose the API over HTTPS) configuration, but I found it much\neasier to not mess with default Docker settings, and to do this by just deploying a container.\n\nBelow configuration `DOCKER_CLIENTCERT` (and its key) refers to the client cert that is allowed to\nconnect to the Docker socket over HTTPS. They can be encoded to base64 like this:\n\n- `$ cat cert.pem | base64 -w 0`\n- `$ cat cert.key | base64 -w 0`\n\n```console\n$ docker service create \\\n\t--name promswarmconnect \\\n\t--env \"DOCKER_URL=https://dockersockproxy:4431\" \\\n\t--env \"DOCKER_CLIENTCERT=...\" \\\n\t--env \"DOCKER_CLIENTCERT_KEY=...\" \\\n\t--env \"NETWORK_NAME=yourNetwork\" \\\n\t--network yourNetwork \\\n\t\"fn61/promswarmconnect:VERSION\"\n```\n\nObviously, you need to replace URL and port with your Docker socket's details.\n\n### Verify that it's working\n\nBefore moving on to configure Prometheus, verify that promswarmconnect is working.\n\nGrab an Alpine container (on the same network), and verify that you can `$ curl` the API:\n\n```console\n$ docker run --rm -it --network yourNetwork alpine sh\n$ apk add curl\n$ curl -k https://promswarmconnect/v1/discover\n{\n  \"containers\": [\n    {\n      \"server_uuid\": \"/metrics\",\n      \"vm_alias\": \"10.0.1.7:8081\",\n      \"vm_brand\": \"http\",\n      \"vm_image_uuid\": \"traefik_traefik\",\n      \"vm_uuid\": \"rsvltiqm6nbcj72ibi7bess0w\"\n    },\n    {\n      \"server_uuid\": \"/metrics\",                 \u003c-- __metrics_path__\n      \"vm_alias\": \"10.0.1.15:80\",                \u003c-- __address__\n      \"vm_brand\": \"http\",                        \u003c-- __scheme__\n      \"vm_image_uuid\": \"hellohttp_hellohttp\",    \u003c-- job (Docker service name)\n      \"vm_uuid\": \"p44b6yr05ucmhpl0teiadq3jt\"     \u003c-- instance (Docker task ID)\n    }\n  ]\n}\n```\n\n[More info here](https://github.com/function61/promswarmconnect/blob/ecc947d4aa6b29bb4595929d2bc23b1ec7bd5e9e/cmd/promswarmconnect/main.go#L207)\non why the JSON keys are so different W.R.T. Prometheus labels they'll be relabeled at\n(see also our config example).\n\n\nRunning in a swarm via docker-compose\n----------------------------------------\n\nYou can launch promswarmconnect via docker-compose, an entry would look similar to the below for the promswarmconnect container.\n```yaml\npromswarmconnect:\n    image: fn61/promswarmconnect:20190126_1620_7b450c47\n    volumes:\n      - /var/run/docker.sock:/var/run/docker.sock\n    environment:\n      - DOCKER_URL=unix:///var/run/docker.sock\n      - NETWORK_NAME=\u003cCHANGETOSTACKNETNAME\u003e\n    deploy:\n      placement:\n        constraints: [node.role == manager]\n\n```\n\nThen for each service you wish to monitor metrics for, add an environment var as noted above in this readme, for example:\n```yaml\nnats_monitoring:\n    image: ainsey11/nats_prometheus_exporter\n    environment:\n      - METRICS_ENDPOINT=:7777/metrics\n    ports:\n      - 7777:7777\n    command: [\"-varz\", \"-connz\", \"-routez\", \"-subz\", \"http://nats:8222\"]\n\n```\n\nExporting per node metrics via multiple containers in a service\n--------------------------------------------------------------\n\nThe prime use case of this is when running something like node_exporter, or cAdvisor as a service with a global constraint\nbasically, each docker host will have a cAdvisor container running, the problem with this is that by default promswarmconnect doesn't return by default, the hostname of the host the container is on\nthis makes it difficult to differentiate which container is which when the data is surfaced into prometheus itself,\n\nas described in Issue Number 4, you can edit the environment variable of the container you wish to autodiscover, as follows:\n``` METRICS_ENDPOINT=/metrics,instance=_HOSTNAME_ ```\n\nthis will then return a value similar to:\n\n```json\n    {\n      \"server_uuid\": \"/metrics\",\n      \"vm_alias\": \"10.0.3.86:8080\",\n      \"vm_brand\": \"http\",\n      \"vm_image_uuid\": \"test_stack1_cadvisor\",\n      \"vm_uuid\": \"nc-docker-1\"\n    },\n\n```\n\n\nConfiguring Prometheus\n----------------------\n\nConfigure your Prometheus:\n[example configuration that works for us](https://github.com/function61/prometheus-conf/blob/e5c70f03674a87f1b1ffed9555f8386a7a84b758/prometheus.yml#L14).\n\n**The relabeling steps are really important**\n\nThe `endpoint` needs to be your service name in Docker that you use to run promswarmconnect.\n\nPro-tip: you could probably use\n[our Prometheus image](https://github.com/function61/prometheus-conf) (check the Docker\nHub link) as-is, if not for production but at least to check out if this concept works for\nyou!\n\n\nConsiderations for running containers\n-------------------------------------\n\npromswarmconnect only picks up containers whose *service-level ENV vars* specify\n`METRICS_ENDPOINT=/metrics`. To use non-80 port, specify `METRICS_ENDPOINT=:8080/metrics`.\nThe metrics path is also configurable, obviously.\n\nFor a complete demo with dummy application, deploy:\n\n- promswarmconnect (instructions were at this document)\n- our prometheus image (instructions were at above pro-tip) and\n- [hellohttp](https://github.com/joonas-fi/hellohttp) (it has built-in Prometheus metrics)\n\n\nFAQ\n---\n\n\u003e Can I read `DOCKER_CLIENTCERT` or `DOCKER_CLIENTCERT_KEY` from file or use Docker secrets?\n\nYes, see [#10](https://github.com/function61/promswarmconnect/issues/10)\n\n\u003e TLS?\n\nSee [#12](https://github.com/function61/promswarmconnect/issues/12#issuecomment-664344435)\n\n\nHow to build \u0026 develop\n----------------------\n\n[How to build \u0026 develop](https://github.com/function61/turbobob/blob/master/docs/external-how-to-build-and-dev.md)\n(with Turbo Bob, our build tool). It's easy and simple!\n\n\nAlternatives \u0026 links\n--------------------\n\n- https://github.com/ContainerSolutions/prometheus-swarm-discovery\n- https://github.com/prometheus/prometheus/issues/1766\n- https://github.com/jmendiara/prometheus-swarm-discovery\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffunction61%2Fpromswarmconnect","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffunction61%2Fpromswarmconnect","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffunction61%2Fpromswarmconnect/lists"}