{"id":23695963,"url":"https://github.com/ziprecruiter/cloudwatching","last_synced_at":"2025-09-02T16:36:26.786Z","repository":{"id":52657226,"uuid":"182288919","full_name":"ZipRecruiter/cloudwatching","owner":"ZipRecruiter","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-16T17:23:17.000Z","size":63,"stargazers_count":8,"open_issues_count":4,"forks_count":7,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-11-17T12:08:25.944Z","etag":null,"topics":["cloudwatch","prometheus"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZipRecruiter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-19T15:49:37.000Z","updated_at":"2023-08-04T02:03:54.000Z","dependencies_parsed_at":"2024-06-19T04:13:35.598Z","dependency_job_id":null,"html_url":"https://github.com/ZipRecruiter/cloudwatching","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZipRecruiter%2Fcloudwatching","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZipRecruiter%2Fcloudwatching/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZipRecruiter%2Fcloudwatching/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZipRecruiter%2Fcloudwatching/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZipRecruiter","download_url":"https://codeload.github.com/ZipRecruiter/cloudwatching/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231819021,"owners_count":18431234,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudwatch","prometheus"],"created_at":"2024-12-30T05:58:22.805Z","updated_at":"2024-12-30T05:58:23.399Z","avatar_url":"https://github.com/ZipRecruiter.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## cloudwatching\n\nThis is a reimplementation of the official\n[cloudwatch-exporter](https://github.com/prometheus/cloudwatch_exporter).  The\nmain motivation is to use GetMetricData instead of GetMetricStatistics, which\ncan lead to a reduction in AWS API calls by a factor of about 100.\n\n## Quickstart\n\n```\ngo get -u github.com/ZipRecruiter/cloudwatching\n```\n\nCopy paste the following config to `~/mc.json`:\n\n```\n{\n  \"exportconfigs\": [\n    {\n      \"dimensions\": [\n        \"QueueName\"\n      ],\n      \"dimensionsMatch\": {\n        \"QueueName\": \"(?i:prod)\"\n      },\n      \"dimensionsNoMatch\": {\n        \"QueueName\": \"(?i:dev|staging)\"\n      },\n      \"name\": \"ApproximateAgeOfOldestMessage\",\n      \"namespace\": \"AWS/SQS\",\n      \"statistics\": [\n        \"Maximum\"\n      ]\n    },\n    {\n      \"dimensions\": [\n        \"QueueName\"\n      ],\n      \"dimensionsMatch\": {\n        \"QueueName\": \"(?i:prod)\"\n      },\n      \"dimensionsNoMatch\": {\n        \"QueueName\": \"(?i:dev|staging)\"\n      },\n      \"name\": \"NumberOfMessagesReceived\",\n      \"namespace\": \"AWS/SQS\",\n      \"statistics\": [\n        \"Maximum\"\n      ]\n    },\n    {\n      \"dimensions\": [\n        \"QueueName\"\n      ],\n      \"dimensionsMatch\": {\n        \"QueueName\": \"(?i:prod)\"\n      },\n      \"dimensionsNoMatch\": {\n        \"QueueName\": \"(?i:dev|staging)\"\n      },\n      \"name\": \"NumberOfMessagesDeleted\",\n      \"namespace\": \"AWS/SQS\",\n      \"statistics\": [\n        \"Maximum\"\n      ]\n    }\n  ],\n  \"region\": \"us-east-1\",\n  \"debug\": false\n}\n```\n\nAnd run:\n\n```bash\nMC_CONFIG=~/mc.json cloudwatching\n```\n\nYou should be able to see the metrics at `locahost:8080`.\n\n## Description\n\nThis tool surfaces AWS CloudWatch metrics as prometheus metrics.  It gets the\nmetric data at scrape time, rather than pulling and caching periodically.\nCurrently the naming of the metrics is not configurable, but that should be easy\nto fix if it would help anyone.\n\nAn important detail to be aware of is that CloudWatch metrics are generally\nunlike prometheus metrics (dimensions are less like labels and more like oddly\nnamed directories.)  For the most part this is not a huge problem, though there\nare caveats:\n\n* for DynamoDB Indexes, we have to append `_index` to the metric to prevent\n  collisions if you are scraping table-level and index-level metrics;\n* for RDS DB Clusters and DB Instances, we have to append `_cluster` and\n  `_instance` respectively to prevent collisions when scraping cluster-level\n  and instance-level metrics.\n\n## Advanced Customization\n\nThe majority of the code for `cloudwatching` is in [a\npackage](https://godoc.org/github.com/ZipRecruiter/cloudwatching/pkg/exportcloudwatch)\nso that less common requirements can be supported by a separate main package.\n\nYou should be able to trivially swap in other configuration styles (like YAML,\nif that's what you prefer,) have prometheus listen at a different location.\n\nIf you do create an alternate version please make sure that you are using go\nmodules or some other pinning strategy.  I have ideas to improve this package\nthat will require breaking changes at some point; if you don't pin, you'll need\nto fix your code when that happens.\n\nI suggest that you look over [how we create the `*cloudwatch.CloudWatch`\nclient](https://github.com/ZipRecruiter/cloudwatching/blob/master/deps.go)\nand copy some of the patterns, since surfacing how the exporter is interacting\nwith the AWS API can be tricky but is worth the effort.\n\n---\n\nHope this helps!\n\nfREW Schmidt\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fziprecruiter%2Fcloudwatching","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fziprecruiter%2Fcloudwatching","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fziprecruiter%2Fcloudwatching/lists"}