{"id":18909459,"url":"https://github.com/gakas14/event-driven-architecture","last_synced_at":"2026-02-22T07:33:54.520Z","repository":{"id":211052166,"uuid":"728076129","full_name":"gakas14/Event-Driven-Architecture","owner":"gakas14","description":"Event driven Architecture with S3, SNS, SQS, and Lambda. Uses Events to decouple an application's components.","archived":false,"fork":false,"pushed_at":"2023-12-06T14:43:09.000Z","size":5,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-24T11:46:13.231Z","etag":null,"topics":["aws-lambda","aws-s3","sns-topic","sqs-queue"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gakas14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-06T07:14:35.000Z","updated_at":"2023-12-06T07:15:32.000Z","dependencies_parsed_at":"2023-12-06T08:29:18.962Z","dependency_job_id":"dd1def30-754d-4445-9160-c217258a7fc9","html_url":"https://github.com/gakas14/Event-Driven-Architecture","commit_stats":null,"previous_names":["gakas14/event-driven-architecture"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gakas14/Event-Driven-Architecture","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gakas14%2FEvent-Driven-Architecture","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gakas14%2FEvent-Driven-Architecture/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gakas14%2FEvent-Driven-Architecture/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gakas14%2FEvent-Driven-Architecture/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gakas14","download_url":"https://codeload.github.com/gakas14/Event-Driven-Architecture/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gakas14%2FEvent-Driven-Architecture/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29706569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T05:59:28.568Z","status":"ssl_error","status_checked_at":"2026-02-22T05:58:46.208Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","aws-s3","sns-topic","sqs-queue"],"created_at":"2024-11-08T09:33:54.460Z","updated_at":"2026-02-22T07:33:54.498Z","avatar_url":"https://github.com/gakas14.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Event-Driven-Architecture\nEvent driven Architecture with S3, SNS, SQS, and Lambda. Uses Events to decouple an application's components.\n- loosely coupled applications\n- add new features without changing existing applications\n- Scale and fail components independently.\n\nI. Create the s3 bucket\n\n\u003cimg width=\"1266\" alt=\"Screen Shot 2023-12-06 at 8 54 44 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/9cc01c6a-4a92-414b-901c-9db2c5245de2\"\u003e\n\n\n\n\nII. Create an SNS topic\n- create the topic and change the access policy: copy your account number, s3 bucket name, and SQS queues.\n  \n\u003cimg width=\"1276\" alt=\"Screen Shot 2023-12-06 at 3 42 37 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/bfcbaa39-aa10-447d-9376-c58fbd17d13c\"\u003e\n\n\n\n\t\t{\n\t  \"Version\": \"2008-10-17\",\n\t  \"Id\": \"__default_policy_ID\",\n\t  \"Statement\": [\n\t    {\n\t      \"Sid\": \"__default_statement_ID\",\n\t      \"Effect\": \"Allow\",\n\t      \"Principal\": {\n\t        \"Service\": \"s3.amazonaws.com\"\n\t      },\n\t      \"Action\": \"SNS:Publish\",\n\t      \"Resource\": \"arn:aws:sns:\u003cregion\u003e:\u003caccountId\u003e:\u003csnsTopicName\u003e\",\n\t      \"Condition\": {\n\t        \"StringEquals\": {\n\t          \"aws:SourceAccount\": \"\u003caccountId\u003e\"\n\t        },\n\t        \"ArnLike\": {\n\t          \"aws:SourceArn\": \"arn:aws:s3:*:*:\u003cs3BucketName\u003e\"\n\t        }\n\t      }\n\t    },\n\t    {\n\t      \"Sid\": \"sqs_statement\",\n\t      \"Effect\": \"Allow\",\n\t      \"Principal\": {\n\t        \"Service\": \"sqs.amazonaws.com\"\n\t      },\n\t      \"Action\": \"sns:Subscribe\",\n\t      \"Resource\": \"arn:aws:sns:\u003cregion\u003e:\u003caccountId\u003e:\u003csnsTopicName\u003e\",\n\t      \"Condition\": {\n\t        \"ArnEquals\": {\n\t          \"aws:SourceArn\": [\n\t            \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:\u003csqsQueueName\u003e\",\n\t            \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:\u003csqsQueueName\u003e\"\n\t          ]\n\t        }\n\t      }\n\t    }\n\t  ]\n\t\t}\n\n\n  \nIII. Create SQS Quere\n\n\u003cimg width=\"1280\" alt=\"Screen Shot 2023-12-06 at 3 53 08 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/5b4eb5a8-4072-459a-a062-1e6d1b59c053\"\u003e\n\n\n\n- event stream: create the queue and change the access policy:\n\n\t  {\n\t  \"Version\": \"2008-10-17\",\n\t  \"Id\": \"__default_policy_ID\",\n\t  \"Statement\": [\n\t    {\n\t      \"Sid\": \"Stmt1234\",\n\t      \"Effect\": \"Allow\",\n\t      \"Principal\": {\n\t        \"Service\": \"lambda.amazonaws.com\"\n\t      },\n\t      \"Action\": [\n\t        \"sqs:ReceiveMessage\",\n\t        \"sqs:sendMessage\"\n\t      ],\n\t      \"Resource\": \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:\u003csqsQueueName\u003e\",\n\t      \"Condition\": {\n\t        \"ArnEquals\": {\n\t          \"aws:SourceArn\": \"arn:aws:lambda:\u003cregion\u003e:\u003caccountId\u003e:\u003clambdaName\u003e\"\n\t        }\n\t      }\n\t    },\n\t    {\n\t      \"Sid\": \"Stmt12345\",\n\t      \"Effect\": \"Allow\",\n\t      \"Principal\": {\n\t        \"AWS\": \"*\"\n\t      },\n\t      \"Action\": \"sqs:SendMessage\",\n\t      \"Resource\": \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:\u003csqsQueueName\u003e\",\n\t      \"Condition\": {\n\t        \"ArnLike\": {\n\t          \"aws:SourceArn\": \"arn:aws:sns:\u003cregion\u003e:\u003caccountId\u003e:\u003csnsTopicName\u003e\"\n\t        }\n\t      }\n\t    }\n\t  ]\n\n\n\nIV. Create the Lambda Functions\n- create two lambda policies to allow access to SQS and CloudWatch\n\n\n\t\t{\n\t\t    \"Version\": \"2012-10-17\",\n\t\t    \"Statement\": [\n\t\t        {\n\t\t            \"Sid\": \"VisualEditor0\",\n\t\t            \"Effect\": \"Allow\",\n\t\t            \"Action\": [\n\t\t                \"sqs:DeleteMessage\",\n\t\t                \"logs:CreateLogStream\",\n\t\t                \"sqs:ReceiveMessage\",\n\t\t                \"sqs:GetQueueAttributes\",\n\t\t                \"logs:PutLogEvents\"\n\t\t            ],\n\t\t            \"Resource\": [\n\t\t                \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:q1\",\n\t\t                \"arn:aws:logs:\u003cregion\u003e:\u003caccountId\u003e:log-group:/aws/lambda/\u003clambdaName\u003e:*\"\n\t\t            ]\n\t\t        },\n\t\t        {\n\t\t            \"Sid\": \"VisualEditor1\",\n\t\t            \"Effect\": \"Allow\",\n\t\t            \"Action\": [\n\t\t                \"sqs:ReceiveMessage\",\n\t\t                \"logs:CreateLogGroup\"\n\t\t            ],\n\t\t            \"Resource\": [\n\t\t                \"arn:aws:logs:\u003cregion\u003e:\u003caccountId\u003e:*\",\n\t\t                \"arn:aws:sqs:\u003cregion\u003e:\u003caccountId\u003e:\u003csqsQueueName\u003e\"\n\t\t            ]\n\t\t        }\n\t\t    ]\n\t\t}\n\n\n\n- Create two roles for each lambda function and then attach each policy to the corresponding lambda role.\n\n\n  \n\u003cimg width=\"1274\" alt=\"Screen Shot 2023-12-06 at 4 29 11 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/a6e149d1-7e77-4935-9152-ff114d7c5234\"\u003e\n\n\n\n- Create the two lambda functions, and let's print the data\n\n\u003cimg width=\"1279\" alt=\"Screen Shot 2023-12-06 at 8 48 06 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/a37e1d04-8196-4c4b-95e8-a62a153bd526\"\u003e\n\n\n\u003cimg width=\"1265\" alt=\"Screen Shot 2023-12-06 at 8 48 35 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/5dd24113-914e-4c34-a4b3-079e12677d55\"\u003e\n\n\nV. Connect the four components\n- connect the s3 bucket to the SNS by creating an event notification at the s3 bucket level: select \"All object create events\" as the event type and the SNS topic as a destination\n\n  \n\u003cimg width=\"1271\" alt=\"Screen Shot 2023-12-06 at 8 55 40 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/95171796-8e87-41db-86cc-a38a4d3bf5c0\"\u003e\n\n\n  \n- connect SQS and the SNS by creating a subscription at the SNS level (with a filter policy). NB: create two subscriptions for each queue.\n  The filter policy: we filter the message based on the event name:\n The put event will be forwarded to queue 1\n\n  \u003cimg width=\"1270\" alt=\"Screen Shot 2023-12-06 at 8 57 19 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/44675871-3ec9-416a-863e-caefa6dfc573\"\u003e\n\n\n\t  {\n\t  \"Records\": {\n\t    \"eventName\": [\n\t      \"ObjectCreated:Put\"\n\t    ]\n\t  }\n\t  }\n\n\n  \n The copy event will be forwarded to queue 2\n\n \n\u003cimg width=\"1277\" alt=\"Screen Shot 2023-12-06 at 9 00 56 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/8d68de81-e951-4de5-a12b-bb311815848b\"\u003e\n\n\t  {\n\t  \"Records\": {\n\t    \"eventName\": [\n\t      \"ObjectCreated:Copy\"\n\t    ]\n\t  }\n\t  }\n  \n- Finally, connect SQS queues to the lambda functions at the SQS level.\n\n\n  \n\u003cimg width=\"1277\" alt=\"Screen Shot 2023-12-06 at 9 05 49 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/16005b50-eeb8-4122-adb3-389d27fa5017\"\u003e\n\n\n\nVI. Test Setup\n- put an  object in the s3 bucket, and let's check the Cloudwatch log into the lambda function 1\n\n\n- \u003cimg width=\"1279\" alt=\"Screen Shot 2023-12-06 at 9 48 59 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/6fb82078-d725-4c70-930e-d5efe0aa2236\"\u003e\n\n\n\u003cimg width=\"1280\" alt=\"Screen Shot 2023-12-06 at 10 07 19 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/993846ee-7d5c-4116-8f44-8a3be1241515\"\u003e\n\n\n\n- copy the same file to check the second lambda log\n\n  \n\u003cimg width=\"1280\" alt=\"Screen Shot 2023-12-06 at 10 07 19 PM\" src=\"https://github.com/gakas14/Event-Driven-Architecture/assets/74584964/b0d97f74-035d-4ad4-af4d-ebe5ef06b882\"\u003e\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgakas14%2Fevent-driven-architecture","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgakas14%2Fevent-driven-architecture","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgakas14%2Fevent-driven-architecture/lists"}