https://github.com/teambion/aws-cw-prometheus-syncer
Metric generator based on the AWS Cloudwatch
https://github.com/teambion/aws-cw-prometheus-syncer
audit aws cloudwatch devops docker eks kubernetes lambda linux
Last synced: 10 months ago
JSON representation
Metric generator based on the AWS Cloudwatch
- Host: GitHub
- URL: https://github.com/teambion/aws-cw-prometheus-syncer
- Owner: TeamBion
- Created: 2021-10-17T10:17:29.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-11-18T15:48:25.000Z (over 3 years ago)
- Last Synced: 2025-04-15T13:54:33.099Z (about 1 year ago)
- Topics: audit, aws, cloudwatch, devops, docker, eks, kubernetes, lambda, linux
- Language: Python
- Homepage:
- Size: 629 KB
- Stars: 12
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[](https://app.travis-ci.com/WoodProgrammer/soprano)
# Cloudwatch Insight Prometheus Exporter
Cloudwatch Insight Prometheus Exporter allows you to generate Prometheus-compatible metrics using AWS Cloudwatch Log Insights.
Many AWS services create logs that you can see in AWS Cloudwatch Logs, making generating metrics impossible.
To be able to generate metrics, we developed this Prometheus Exporter so we can generate metrics from Cloudwatch Logs using Cloudwatch Log Insights.
Table of contents
- [Getting started](#getting-started)
- [Deployment](#deploy)
- [Configuration](#configuration)
## Getting Started
* Quick Start - EKS Audit logs
## Deployment
Helm chart is located in deploy/soprano directory and you can easily setup the helm chart shown at below.
```sh
$ export BASE_PATH=$(PWD)/deploy/soprano
$ helm upgrade -i aws-cw-prometheus-syncer ${BASE_PATH} -f ${BASE_PATH}/values.yaml
```
### Caveats
To make this tool able to fetch the results of the CloudwatchInsights queries you should make sure the permissions are setup true.
### Permissions
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "LogGroupRetention",
"Effect": "Allow",
"Action": [
"logs:Get*",
"logs:Describe*",
"logs:StartQuery",
"logs:StopQuery",
"logs:GetQueryResults",
"cloudwathc:Get*"
]
"Resource": "arn:aws:logs:REGION:ACCOUNT_ID:log-group:LOG-GROUP-NAME"
}
]
}
```
This tools is also able to access AWS API via IRSA [Recommended way] assumed way so basically you can setup the IRSA like that;
```yaml
serviceAccount:
create: true
annotations:
eks.amazonaws.com/role-arn: ROLE_ARN
```
## Configuration
You can check the helm configuration details overs there ;
| Parameter | Description | Default |
| --------------------------------- | ----------------------------------------------------------------------- | --------------------------- |
| `image.repository` | Image | `emirozbir/soprano
| `eks.region` | The aws region where you are working | `eu-west-1
| `eks.log_group_name` | Log group address to work on | `/aws/eks/test-cluster/cluster
| `servicemonitor.exporter_key` | Release name of the kube-prometheus stack | `eu-west-1
| `servicemonitor.namespace` | Prometheus operator namespace on | `eu-west-1
| `exporter.port` | Exposed port value on | `9877
| `serviceAccount.annotations` | Annocations for the IRSA usage or generic purpose staff on | `eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/ROLE-FOR-SOPRANO"
Important:
To show respect to the James Gandolfini I keep the helm chart name as Soprano.