{"id":17081486,"url":"https://github.com/benfoster/peekaqueue","last_synced_at":"2026-04-29T11:02:29.699Z","repository":{"id":55515968,"uuid":"184128066","full_name":"benfoster/peekaqueue","owner":"benfoster","description":"Drop-in docker container for monitoring SQS queues","archived":false,"fork":false,"pushed_at":"2020-08-31T11:45:56.000Z","size":21,"stargazers_count":2,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-22T19:59:48.262Z","etag":null,"topics":["sqs"],"latest_commit_sha":null,"homepage":null,"language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benfoster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-29T19:06:02.000Z","updated_at":"2021-02-08T17:16:13.000Z","dependencies_parsed_at":"2022-08-15T02:10:46.017Z","dependency_job_id":null,"html_url":"https://github.com/benfoster/peekaqueue","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/benfoster/peekaqueue","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benfoster%2Fpeekaqueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benfoster%2Fpeekaqueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benfoster%2Fpeekaqueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benfoster%2Fpeekaqueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benfoster","download_url":"https://codeload.github.com/benfoster/peekaqueue/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benfoster%2Fpeekaqueue/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32422532,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T06:29:02.080Z","status":"ssl_error","status_checked_at":"2026-04-29T06:29:00.631Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["sqs"],"created_at":"2024-10-14T12:53:16.958Z","updated_at":"2026-04-29T11:02:29.665Z","avatar_url":"https://github.com/benfoster.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"![GitHub release](https://img.shields.io/github/release/benfoster/peekaqueue.svg)\n![Docker Cloud Build Status](https://img.shields.io/docker/cloud/build/benfoster/peekaqueue.svg)\n![Docker Pulls](https://img.shields.io/docker/pulls/benfoster/peekaqueue.svg) \n\n# peekaqueue\n\nPeekaqueue provides a drop-in docker container for monitoring SQS queues and enabling queue-based auto-scaling of ECS services.\n\nUnlike the default SQS metrics which are only published to CloudWatch every 5 minutes, peekaqueue can be configured to gather queue metrics much more frequently.\n\n## Features\n\n- Gets SQS queue metrics and exposes them via a [Prometheus](https://prometheus.io/) metrics endpoint\n- Optionally retrieves the desired/pending/running counts of queue consumers running in ECS\n- Optionally publishes the \"backlog-per-instance\" for your queue consumers to CloudWatch in order to configure auto-scaling\n\n### Prometheus metrics\n\nPeekaqueue can be used as a [Prometheus](https://prometheus.io/) exporter. By default, the following SQS queue attributes are exposed as metrics at http://localhost:5000/metrics:\n\n| SQS Attribute                           | Metric Name                      | Labels  |\n|-----------------------------------------|----------------------------------|---------|\n| `ApproximateNumberOfMessages`           | `sqs_available_messages_count`   | `queue` |\n| `ApproximateNumberOfMessagesDelayed`    | `sqs_delayed_messages_count`     | `queue` |\n| `ApproximateNumberOfMessagesNotVisible` | `sqs_not_visible_messages_count` | `queue` |\n\n### Monitoring Queue Consumers\n\nOne of the main goals of peekaqueue is to also monitor the consumers of SQS queues. It assumes that consumers are dockerized applications running in ECS. When configured, the following Prometheus metrics are exposed:\n\n| ECS Service Attribute | Metric Name                 | Labels                       |\n|-----------------------|-----------------------------|------------------------------|\n| `DesiredCount`        | `ecs_service_desired_count` | `ecs_cluster`, `ecs_service` |\n| `PendingCount`        | `ecs_service_pending_count` | `ecs_cluster`, `ecs_service` |\n| `RunningCount`        | `ecs_service_running_count` | `ecs_cluster`, `ecs_service` |\n\n### Calculating Consumer Backlogs\n\nTo auto-scale queue consumers based on the number of messages, we need to determine the backlog-per-instance, that is how many messages are available per consumer instance (`sqs_available_messages_count/ecs_service_running_count`).\n\nThis value is exposed as the following Prometheus metric:\n\n| Metric Name                 | Labels                                |\n|-----------------------------|---------------------------------------|\n| `ecs_service_backlog_count` | `ecs_cluster`, `ecs_service`, `queue` |\n\nIn order to use this metric for auto-scaling it is published to CloudWatch as `EcsServiceBacklog` under the `Custom` metrics namespace (configurable). It includes the following CloudWatch metric dimensions:\n\n- Queue\n- EcsCluster\n- EcsService\n\n## Configuration\n\nIn most cases peekaqueue can be deployed direct to ECS using the images hosted on [Docker Hub](https://hub.docker.com/r/benfoster/peekaqueue) and configured using the following environment variables:\n\n| Environment Variable                                         | Data Type | Default Value           | Description                                                                                               |\n|--------------------------------------------------------------|-----------|-------------------------|-----------------------------------------------------------------------------------------------------------|\n| `PEEKAQ_Monitoring__IntervalInSeconds`                       | `int`     | `15`                    | The interval in seconds in which peekaqueue will collect stats about your queues and consumers            |\n| `PEEKAQ_Monitoring__MetricsEndpointPort`                     | `int`     | `5000`                  | The port that the prometheus metrics will run on                                                          |\n| `PEEKAQ_Monitoring__MetricsEndpointPath`                     | `string`  | `metrics/`              | The path of the metrics endpoint. Must end in a trailing `/`                                              |\n| `PEEKAQ_Monitoring__CloudWatchNamespace`                     | `string`  | `Custom`                | The CloudWatch namespace for publishing metrics to                                                        |\n| `AWS_Region`                                                 | `string`  |                         | The AWS region in which your resources reside e.g. `eu-west-2`                                            |\n| `PEEKAQ_Monitoring__Queues__X__Name`                         | `string`  |                         | The name of the queue you wish to monitor. `X` is the 0-based array indexer of the queues.                |\n| `PEEKAQ_Monitoring__Queues__X__ConsumerCluster`              | `string`  |                         | The name of the ECS cluster in which your queue consumer resides                                          |\n| `PEEKAQ_Monitoring__Queues__X__ConsumerService`              | `string`  |                         | The name of the ECS service in which your queue consumer resides                                          |\n| `PEEKAQ_Serilog__WriteTo__0__Args__restrictedToMinimumLevel` | `string`  | `Information`           | Configures the console logging level. It it recommended to set this to `Error` when running in production |\n| `PEEKAQ_Serilog__WriteTo__1__Args__restrictedToMinimumLevel` | `string`  | `Information`           | If using [Seq](https://datalust.co/seq logging), configures the Seq logging level                         |\n| `PEEKAQ_Serilog__WriteTo__1__Args__serverUrl`                | `string`  | `http://localhost:5341` | The Seq server URL                                                                                        |\n| `PEEKAQ_Serilog__WriteTo__1__Args__apiKey`                   | `string`  |                         | API key for authenticating with Seq                                                                       |\n\nIf you are building the image from scratch you can instead update `appsettings.json` as required.\n\n## AWS Permissions and Credentials\n\nIt is assumed that you will run peekaqueue as an ECS service which should be given an ECS task role with the following permissions:\n\n- sqs:GetQueueUrl\n- sqs:GetQueueAttributes\n- cloudwatch:PutMetricData\n- ecs:DescribeServices\n\nBelow is an example policy that provides access to a single queue:\n\n```\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Sid\": \"VisualEditor0\",\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"sqs:GetQueueUrl\",\n                \"sqs:GetQueueAttributes\"\n            ],\n            \"Resource\": \"arn:aws:sqs:eu-west-2:*:my-queue-name\"\n        },\n        {\n            \"Sid\": \"VisualEditor1\",\n            \"Effect\": \"Allow\",\n            \"Action\": [\n                \"cloudwatch:PutMetricData\",\n                \"ecs:DescribeServices\"\n            ],\n            \"Resource\": \"*\"\n        }\n    ]\n}\n```\n\n## Running Locally\n\nTo test peekaqueue on your local machine against AWS, you must provide your AWS access keys:\n\n```\ndocker run --rm -it \\\n-e AWS_REGION=eu-west-2 \\\n-e AWS_ACCESS_KEY_ID=YOURKEYID \\\n-e AWS_SECRET_ACCESS_KEY=YOURKEY \\\n-e \"PEEKAQ_MONITORING__QUEUES__0__Name=QUEUENAME\" \\\n-p 5000:5000 benfoster/peekaqueue:latest\n```\n\n## Developing peekaqueue\n\nPeekqueue is a .NET Core 2.2 application. You can download the .NET Core runtime and tools [here](http://dot.net).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenfoster%2Fpeekaqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenfoster%2Fpeekaqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenfoster%2Fpeekaqueue/lists"}