{"id":23496076,"url":"https://github.com/teambion/aws-cw-prometheus-syncer","last_synced_at":"2025-08-24T06:16:14.084Z","repository":{"id":39744230,"uuid":"418090369","full_name":"TeamBion/aws-cw-prometheus-syncer","owner":"TeamBion","description":"Metric generator based on the AWS Cloudwatch","archived":false,"fork":false,"pushed_at":"2022-11-18T15:48:25.000Z","size":644,"stargazers_count":12,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-15T13:54:33.099Z","etag":null,"topics":["audit","aws","cloudwatch","devops","docker","eks","kubernetes","lambda","linux"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TeamBion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-17T10:17:29.000Z","updated_at":"2023-01-27T23:42:44.000Z","dependencies_parsed_at":"2023-01-22T09:45:45.632Z","dependency_job_id":null,"html_url":"https://github.com/TeamBion/aws-cw-prometheus-syncer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeamBion%2Faws-cw-prometheus-syncer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeamBion%2Faws-cw-prometheus-syncer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeamBion%2Faws-cw-prometheus-syncer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TeamBion%2Faws-cw-prometheus-syncer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TeamBion","download_url":"https://codeload.github.com/TeamBion/aws-cw-prometheus-syncer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249085481,"owners_count":21210267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audit","aws","cloudwatch","devops","docker","eks","kubernetes","lambda","linux"],"created_at":"2024-12-25T04:10:53.725Z","updated_at":"2025-04-15T13:54:41.784Z","avatar_url":"https://github.com/TeamBion.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Pylint Status](https://app.travis-ci.com/WoodProgrammer/eks-cloudwatch-audit.svg?branch=hot_fix_1)](https://app.travis-ci.com/WoodProgrammer/soprano)\n\n# Cloudwatch Insight Prometheus Exporter\n\nCloudwatch Insight Prometheus Exporter allows you to generate Prometheus-compatible metrics using AWS Cloudwatch Log Insights.\n\nMany AWS services create logs that you can see in AWS Cloudwatch Logs, making generating metrics impossible. \n\nTo be able to generate metrics, we developed this Prometheus Exporter so we can generate metrics from Cloudwatch Logs using Cloudwatch Log Insights.\n\n\n\u003cimg height=\"350\" width=\"1000\" src=\"./img/diagram.png\"\u003e\u003c/img\u003e\n\n\u003cb\u003e Table of contents \u003c/b\u003e\n\n- [Getting started](#getting-started)\n  - [Deployment](#deploy)\n- [Configuration](#configuration)\n\n## Getting Started\n\n* \u003ca href=\"./deploy/soprano/values.yaml\"\u003eQuick Start - EKS Audit logs\u003c/a\u003e\n\n## Deployment\n\nHelm chart is located in deploy/soprano directory and you can easily setup the helm chart shown at below.\n\n```sh\n  $ export BASE_PATH=$(PWD)/deploy/soprano\n  \n  $ helm upgrade -i aws-cw-prometheus-syncer ${BASE_PATH} -f ${BASE_PATH}/values.yaml\n```\n\n### Caveats \nTo make this tool able to fetch the results of the CloudwatchInsights queries you should make sure the permissions are setup true.\n\n### Permissions\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n      {\n        \"Sid\": \"LogGroupRetention\",\n        \"Effect\": \"Allow\",\n        \"Action\": [\n            \"logs:Get*\",\n            \"logs:Describe*\",\n            \"logs:StartQuery\",\n            \"logs:StopQuery\",\n            \"logs:GetQueryResults\",\n            \"cloudwathc:Get*\"\n        ]\n        \"Resource\": \"arn:aws:logs:REGION:ACCOUNT_ID:log-group:LOG-GROUP-NAME\"\n      }\n  ]\n}\n```\n\nThis tools is also able to access AWS API via  IRSA [Recommended way] assumed way so basically you can setup the IRSA like that;\n\n```yaml\nserviceAccount:\n  create: true\n  annotations:\n    eks.amazonaws.com/role-arn: ROLE_ARN\n```\n\n## Configuration\n\nYou can check the helm configuration details overs there ; \n\n| Parameter                         | Description                                                             | Default                     |\n| --------------------------------- | ----------------------------------------------------------------------- | --------------------------- |\n| `image.repository`                | Image                                                                   | `emirozbir/soprano\n| `eks.region`                | The aws region where you are working                                                                    | `eu-west-1\n| `eks.log_group_name`                | Log group address to work on                                                                    | `/aws/eks/test-cluster/cluster\n| `servicemonitor.exporter_key`                |  Release name of the kube-prometheus stack                                                                     | `eu-west-1\n| `servicemonitor.namespace`                | Prometheus operator namespace on                                                                   | `eu-west-1\n| `exporter.port`                | Exposed port value on                                                                   | `9877\n| `serviceAccount.annotations`                | Annocations for the IRSA usage or generic purpose staff on                                                                   | `eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/ROLE-FOR-SOPRANO\"\n\n\u003cb\u003eImportant:\u003c/b\u003e\n\nTo show respect to the James Gandolfini I keep the helm chart name as Soprano.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteambion%2Faws-cw-prometheus-syncer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fteambion%2Faws-cw-prometheus-syncer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteambion%2Faws-cw-prometheus-syncer/lists"}