{"id":29023575,"url":"https://github.com/thesmartmonkey/aws-cdk-fanout","last_synced_at":"2026-03-06T08:32:58.269Z","repository":{"id":291654700,"uuid":"978320713","full_name":"TheSmartMonkey/aws-cdk-fanout","owner":"TheSmartMonkey","description":"CDK contruct for a Fanout pattern with Api Gateway \u003e SNS \u003e SQS \u003e lambda","archived":false,"fork":false,"pushed_at":"2025-05-11T14:11:52.000Z","size":277,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-30T00:30:59.083Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TheSmartMonkey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-05T20:01:16.000Z","updated_at":"2025-05-11T14:11:54.000Z","dependencies_parsed_at":"2025-05-05T21:40:03.601Z","dependency_job_id":"ef30adb2-1636-4514-b660-61c236a29044","html_url":"https://github.com/TheSmartMonkey/aws-cdk-fanout","commit_stats":null,"previous_names":["thesmartmonkey/aws-cdk-fanout"],"tags_count":1,"template":false,"template_full_name":"TheSmartMonkey/create-typescript-npm-library","purl":"pkg:github/TheSmartMonkey/aws-cdk-fanout","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheSmartMonkey%2Faws-cdk-fanout","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheSmartMonkey%2Faws-cdk-fanout/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheSmartMonkey%2Faws-cdk-fanout/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheSmartMonkey%2Faws-cdk-fanout/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TheSmartMonkey","download_url":"https://codeload.github.com/TheSmartMonkey/aws-cdk-fanout/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheSmartMonkey%2Faws-cdk-fanout/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30167963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T07:56:45.623Z","status":"ssl_error","status_checked_at":"2026-03-06T07:55:55.621Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-26T03:30:58.744Z","updated_at":"2026-03-06T08:32:58.250Z","avatar_url":"https://github.com/TheSmartMonkey.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS CDK Fanout Construct\n\nCDK contruct for a Fanout pattern with Api Gateway \u003e SNS \u003e SQS \u003e lambda\n\n[![npm version](https://badge.fury.io/js/aws-cdk-fanout.svg)](https://badge.fury.io/js/aws-cdk-fanout)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n## Overview\n\nThe AWS CDK Fanout construct simplifies creating webhook fanout architectures in AWS. It provides a streamlined way to:\n\n1. Receive webhook events via API Gateway\n2. Publish events to an SNS topic\n3. Route messages to multiple SQS queues based on filtering criteria\n4. Process messages with Lambda functions\n\n![Architecture Diagram](https://github.com/TheSmartMonkey/aws-cdk-fanout/blob/main/.github/images/achi.png)\n\n## Installation\n\n```bash\nnpm install aws-cdk-fanout\n```\n\n## Getting Started\n\n### Basic Usage\n\n```typescript\nimport { FanoutConstruct, FanoutConstructPropsEntity, SqsToLambdaPropsEntity } from 'aws-cdk-fanout';\nimport { Duration, Stack } from 'aws-cdk-lib';\nimport { FilterOrPolicy } from 'aws-cdk-lib/aws-sns';\nimport { Construct } from 'constructs';\n\nexport class MyStack extends Stack {\n  constructor(scope: Construct, id: string) {\n    super(scope, id);\n    \n    // Define the SQS to Lambda mapping\n    const orderProcessingQueue = new SqsToLambdaPropsEntity({\n      lambdaName: 'process-orders',\n      handlerPath: 'src/lambdas/process-orders.ts',\n      envVars: {\n        ENVIRONMENT: 'dev',\n      },\n      snsFilter: {\n        eventType: FilterOrPolicy.stringFilter({\n          allowlist: ['order.created', 'order.updated']\n        })\n      },\n      sqsMaxBatchSize: 10,\n      sqsMaxBatchingWindow: Duration.seconds(30),\n      sqsVisibilityTimeout: Duration.seconds(120)\n    });\n    \n    // Create the fanout construct\n    new FanoutConstruct(this, 'WebhooksFanout', new FanoutConstructPropsEntity({\n      stage: 'dev',\n      region: 'us-east-1',\n      sqsToLambda: [orderProcessingQueue]\n    }));\n  }\n}\n```\n\n### Lambda Handler Example\n\nHere's an example of a Lambda handler that processes messages from the SQS queue:\n\n```typescript\nimport { SQSEvent } from 'aws-lambda';\n\nexport const handler = async (event: SQSEvent): Promise\u003cvoid\u003e =\u003e {\n  for (const record of event.Records) {\n    console.log('Processing message:', record.body);\n    \n    // Parse the message body\n    const message = JSON.parse(record.body);\n    const snsMessage = JSON.parse(message.Message);\n    \n    // Process the message based on your business logic\n    console.log('Event data:', snsMessage);\n  }\n};\n```\n\n## Features\n\n- **API Gateway Integration**: Automatically creates an API Gateway endpoint to receive webhook events\n- **Message Filtering**: Filter messages using SNS message filtering attributes\n- **Dead Letter Queues**: Automatic DLQ setup for failed message processing\n- **Batch Processing**: Configure batch size and batching window for Lambda processing\n- **Secure by Default**: Includes API key authentication for the API Gateway endpoint\n\n## Advanced Configuration\n\n### Multiple Consumers with Different Filters\n\n```typescript\n// Order processing queue\nconst orderProcessingQueue = new SqsToLambdaPropsEntity({\n  lambdaName: 'process-orders',\n  handlerPath: 'src/lambdas/process-orders.ts',\n  envVars: { /* ... */ },\n  snsFilter: {\n    eventType: FilterOrPolicy.stringFilter({\n      allowlist: ['order.created', 'order.updated']\n    })\n  },\n  sqsMaxBatchSize: 10,\n  sqsMaxBatchingWindow: Duration.seconds(30),\n  sqsVisibilityTimeout: Duration.seconds(120)\n});\n\n// User processing queue\nconst userProcessingQueue = new SqsToLambdaPropsEntity({\n  lambdaName: 'process-users',\n  handlerPath: 'src/lambdas/process-users.ts',\n  envVars: { /* ... */ },\n  snsFilter: {\n    eventType: FilterOrPolicy.stringFilter({\n      allowlist: ['user.created', 'user.updated']\n    })\n  },\n  sqsMaxBatchSize: 5,\n  sqsMaxBatchingWindow: Duration.seconds(60),\n  sqsVisibilityTimeout: Duration.seconds(180)\n});\n\n// Create the fanout construct with multiple consumers\nnew FanoutConstruct(this, 'WebhooksFanout', new FanoutConstructPropsEntity({\n  stage: 'dev',\n  region: 'us-east-1',\n  sqsToLambda: [orderProcessingQueue, userProcessingQueue]\n}));\n```\n\n### Customizing Queue Options\n\n```typescript\nconst queue = new SqsToLambdaPropsEntity({\n  // ... other properties\n  queueOptions: {\n    retentionPeriod: Duration.days(7),\n    dataKeyReuse: Duration.hours(1),\n    encryption: sqs.QueueEncryption.KMS_MANAGED\n  }\n});\n```\n\n### Customizing Lambda Options\n\n```typescript\nconst queue = new SqsToLambdaPropsEntity({\n  // ... other properties\n  lambdaOptions: {\n    memorySize: 512,\n    timeout: Duration.seconds(30),\n    runtime: Runtime.NODEJS_18_X\n  }\n});\n```\n\n### Turning Off Components\n\n```typescript\nnew FanoutConstruct(this, 'WebhooksFanout', new FanoutConstructPropsEntity({\n  stage: 'dev',\n  region: 'us-east-1',\n  sqsToLambda: [...],\n  removeApiGateway: true,          // Don't create an API Gateway\n  removeApiGatewayKeyAuth: true,   // Don't use API key authentication\n  removeLambda: true               // Don't create Lambda functions\n}));\n```\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](https://github.com/TheSmartMonkey/aws-cdk-fanout/blob/main/LICENSE) file for details.\n\n## Testing\n\n### Testing the Fanout Filtering\n\nThe construct supports filtering messages based on the `eventType` field. You can test this functionality using the provided test:\n\n```typescript\nimport { LocalStackClient } from './tests/localstack';\n\n// Initialize LocalStack\nconst localStackClient = await LocalStackClient.getInstance();\nawait localStackClient.initStack();\n\n// Get the filter configurations\nconst sendEventFilterConfig = localStackClient.getFilterConfig('send-event');\nconst receiveEventFilterConfig = localStackClient.getFilterConfig('receive-event');\n\n// Verify that messages are routed correctly based on eventType\nconst message = { \n  eventType: 'send',\n  data: { /* your payload */ }\n};\n\n// In a real scenario, you would send this message to the API Gateway endpoint:\n// POST /send-event with the message as the body\n```\n\nWhen a message with `eventType: \"send\"` is sent to the API Gateway, it will be published to the SNS topic, and then filtered to the appropriate SQS queue that triggers the Lambda with the `send-event` name.\n\n### Running E2E Tests\n\nTo run the end-to-end tests with LocalStack:\n\n```bash\nnpm test tests/e2e/api-gateway.e2e.test.ts\n```\n\nThis test verifies that messages with `eventType: \"send\"` are correctly routed to the Lambda function named `send-event`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthesmartmonkey%2Faws-cdk-fanout","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthesmartmonkey%2Faws-cdk-fanout","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthesmartmonkey%2Faws-cdk-fanout/lists"}