{"id":16769520,"url":"https://github.com/cmur2/promgen-k8s","last_synced_at":"2025-03-21T23:34:01.098Z","repository":{"id":25186772,"uuid":"103374324","full_name":"cmur2/promgen-k8s","owner":"cmur2","description":"A modular Prometheus 2 configuration file generator to monitor multiple Kubernetes clusters with a single Prometheus instance.","archived":false,"fork":false,"pushed_at":"2023-12-07T00:50:20.000Z","size":75,"stargazers_count":9,"open_issues_count":3,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-18T06:11:25.658Z","etag":null,"topics":["aws","configuration","generator","kops","kubernetes","prometheus"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cmur2.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-13T08:29:56.000Z","updated_at":"2024-09-15T16:53:21.000Z","dependencies_parsed_at":"2024-10-28T11:40:50.043Z","dependency_job_id":"9b7eb717-bd83-4f29-bdd5-7dcba2b69e5a","html_url":"https://github.com/cmur2/promgen-k8s","commit_stats":{"total_commits":99,"total_committers":4,"mean_commits":24.75,"dds":"0.29292929292929293","last_synced_commit":"02207b595359100387d7645eb4d3beedd665ad32"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmur2%2Fpromgen-k8s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmur2%2Fpromgen-k8s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmur2%2Fpromgen-k8s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmur2%2Fpromgen-k8s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cmur2","download_url":"https://codeload.github.com/cmur2/promgen-k8s/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244885398,"owners_count":20526290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","configuration","generator","kops","kubernetes","prometheus"],"created_at":"2024-10-13T06:14:18.563Z","updated_at":"2025-03-21T23:34:00.787Z","avatar_url":"https://github.com/cmur2.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# promgen-k8s\n\n![ci](https://github.com/cmur2/promgen-k8s/workflows/ci/badge.svg)\n\nA modular [Prometheus 2](https://prometheus.io) configuration file generator to monitor multiple [Kubernetes](https://kubernetes.io) clusters with a single Prometheus instance.\n\n## Scenario\n\nYou have one Kubernetes cluster per environment (dev, qa, staging, production, etc.) for whichever reasons including e.g. special permission model, limiting the blast radius of problems related to Kubernetes or to test Kubernetes upgrades in staging environment first.\nYou have a separate Kubernetes cluster for operations that runs your monitoring and logging setup, CI pipeline, etc.\nYou want to monitor all environments/clusters with a single Prometheus instance located in the operations cluster.\n\nWriting, modifying and maintaining the [prometheus.yml](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml) for one Kubernetes cluster by hand is tedious and error prone. Doing so for multiple clusters that should share most of the configuration except for specific rules can become a nightmare.\n\npromgen-k8s aims to provide a toolkit allowing you to easily generate a prometheus.yml tailored for monitoring multiple Kubernetes clusters with invididual scraping jobs and relabeling rules per cluster.\n\n## Current State\n\npromgen-k8s is currently in a very early state where it manages to generate the example  [prometheus-kubernetes.yml](https://github.com/prometheus/prometheus/blob/master/documentation/examples/prometheus-kubernetes.yml) for multiple clusters using [autodiscovery](https://prometheus.io/docs/operating/configuration/#%3Ckubernetes_sd_config%3E) including custom relabeling rules per cluster. It assumes that it can reach all Kubernetes apiservers by predictable DNS names using the provided secret files for authentication.\n\nFor ingresses, pod, service and service endpoint monitoring in remote clusters promgen-k8s uses the Kubernetes apiservers as an [HTTP proxy](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-services/).\n\nAs ingresses and services are checked using the [blackbox-exporter](https://github.com/prometheus/blackbox_exporter/) promgen-k8s also assumes a `blackbox-exporter` pod and service in `monitoring` namespace running in each cluster (including remote clusters).\n\npromgen-k8s is successfully used with Kubernetes 1.8 clusters on [AWS](http://aws.amazon.com/) created by [kops](https://github.com/kubernetes/kops).\n\n## Example\n\nThe [example](example-generator.py) uses a [stub file](example-prometheus-stub.yml) for manual configuration and can be tested using `pip install promgen-k8s` and then `python example-generator.py`.\n\n## Related Work\n\n- [https://github.com/line/promgen]([https://github.com/line/promgen])\n\n## Doing a Release\n\nYou need to have the [Poetry](https://python-poetry.org/docs/) dependency manager installed and a [PyPI](https://pypi.org/) account (a [test account](https://test.pypi.org/) works as well) including an authentication token.\n\n### Testing the Release\n\nOne-time setup:\n\n```bash\npoetry config repositories.testpypi https://test.pypi.org/legacy/\npoetry config pypi-token.testpypi TOKEN_FROM_TESTPYPI_ACCOUNT\n```\n\nBump version in [pyproject.toml](pyproject.toml), update the [changelog](CHANGELOG.md) with new release and date and commit.\n\nBuild and release:\n\n```bash\npoetry publish -r testpypi --build\n```\n\nCheck installation:\n\n```bash\npip install --index-url https://test.pypi.org/simple/ promgen_k8s\n```\n\n### Official Release\n\nOne-time setup:\n\n```bash\npoetry config pypi-token.pypi TOKEN_FROM_PYPI_ACCOUNT\n```\n\nBump version in [pyproject.toml](pyproject.toml), update the [changelog](CHANGELOG.md) with new release and date and commit.\n\nBuild and release:\n\n```bash\npoetry publish --build\n```\n\nCheck installation:\n\n```bash\npip install promgen_k8s\n```\n\n## License\n\npromgen-k8s is licensed under the Apache License, Version 2.0. See LICENSE for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcmur2%2Fpromgen-k8s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcmur2%2Fpromgen-k8s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcmur2%2Fpromgen-k8s/lists"}