{"id":19274040,"url":"https://github.com/sbstjn/sqs-worker-serverless","last_synced_at":"2025-04-21T22:33:30.743Z","repository":{"id":56468032,"uuid":"85000185","full_name":"sbstjn/sqs-worker-serverless","owner":"sbstjn","description":"Example for SQS Worker in AWS Lambda using Serverless","archived":false,"fork":false,"pushed_at":"2018-07-12T09:45:43.000Z","size":1531,"stargazers_count":176,"open_issues_count":2,"forks_count":38,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-01T16:24:22.226Z","etag":null,"topics":["aws","dynamodb","lambda","serverless","sqs","worker"],"latest_commit_sha":null,"homepage":"https://sbstjn.com/serverless-sqs-worker-with-aws-lambda.html","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbstjn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-14T21:44:16.000Z","updated_at":"2025-03-22T21:41:27.000Z","dependencies_parsed_at":"2022-08-15T19:20:26.890Z","dependency_job_id":null,"html_url":"https://github.com/sbstjn/sqs-worker-serverless","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbstjn%2Fsqs-worker-serverless","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbstjn%2Fsqs-worker-serverless/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbstjn%2Fsqs-worker-serverless/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbstjn%2Fsqs-worker-serverless/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbstjn","download_url":"https://codeload.github.com/sbstjn/sqs-worker-serverless/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250145371,"owners_count":21382403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","dynamodb","lambda","serverless","sqs","worker"],"created_at":"2024-11-09T20:44:55.125Z","updated_at":"2025-04-21T22:33:26.235Z","avatar_url":"https://github.com/sbstjn.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SQS Worker with Serverless ⚡️\n\n[![MIT License](https://badgen.now.sh/badge/License/MIT/blue)](https://github.com/sbstjn/sqs-worker-serverless/blob/master/LICENSE.md)\n[![Read Tutorial](https://badgen.now.sh/badge/Read/Tutorial/orange)](https://sbstjn.com/serverless-sqs-worker-with-aws-lambda.html)\n\n\nExperimental [Serverless Amazon SQS Worker with AWS Lambda](https://sbstjn.com/serverless-sqs-worker-with-aws-lambda.html).\n\n***Update:** Please see [Serverless Analytics](https://github.com/sbstjn/serverless-analytics) for a more realistic setup using Amazon Kinesis Streams to process your data and events. You can read more about the Kinesis and Lambda connection at [Serverless Analytics with Amazon Kinesis and AWS Lambda](https://sbstjn.com/serverless-analytics-with-kinesis-stream-lambda.html). This was only intended as an April Fools' project …* \n\n## Setup\n\n- SQS Queue with your messages\n- SNS Topic to handle CloudWatch Alarms\n- DynamoDB table to persist configuration\n- CloudWatch Schedule as cron replacement\n- Three (`scale`, `worker`, `process`) AWS Lambda functions\n\n## Workflow\n\n- CloudWatch Alarms on queue length post to SNS\n- SNS Topic triggers `scale` Lambda function\n- Function `scale` updates configuration in DynamoDB \n- CloudWatch Schedule invokes `worker` every `x` minute(s)\n- Function `worker` reads configuration from DynamoDB\n- Function `worker` invokes `process` function(s)\n\n## Auto-Scaling with CloudWatch Alerts\n\n![](./docs/scale.png) \n\n## Workers with CloudWatch Schedule\n\n![](./docs/worker.png)\n\n## Deploy\n\n```bash\n$ \u003e yarn install\n$ \u003e yarn deploy\n```\n\n## Add noise to SQS\n\nYou should have some data in your queue to test this setup. Use [wrk](https://github.com/wg/wrk) to send messages to SQS, but make sure to enable [anonymous access to sendMessage](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/acp-overview.html#anonQueues) for your queue first!\n\n```\n$ \u003e wrk -c35 -d60 -t35 \\\n    -s helpers/wrk.lua \\\n    https://sqs.REGION.amazonaws.com/ACCOUNT-ID/YourQueueName\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbstjn%2Fsqs-worker-serverless","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbstjn%2Fsqs-worker-serverless","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbstjn%2Fsqs-worker-serverless/lists"}