{"id":19854645,"url":"https://github.com/obytes/terraform-aws-sumu","last_synced_at":"2025-11-24T11:06:25.548Z","repository":{"id":77139369,"uuid":"428186265","full_name":"obytes/terraform-aws-sumu","owner":"obytes","description":"Reusable and Pluggable Websocket Stack that can be hooked to any application to make it interactive and provide realtime capability.","archived":false,"fork":false,"pushed_at":"2021-11-21T11:41:06.000Z","size":210,"stargazers_count":7,"open_issues_count":0,"forks_count":2,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-01-11T13:50:41.832Z","etag":null,"topics":["aws-apigateway","interactive","jwt","lambda","realtime","serverless","sns","sqs","websocket"],"latest_commit_sha":null,"homepage":"https://www.obytes.com/blog/go-serverless-part-4-realtime-interactive-and-secure-applications-with-aws-websocket-api-gateway","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/obytes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-15T08:46:17.000Z","updated_at":"2023-09-13T08:10:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"c5837916-6b8b-4c94-aa61-ab12e1f2db41","html_url":"https://github.com/obytes/terraform-aws-sumu","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obytes%2Fterraform-aws-sumu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obytes%2Fterraform-aws-sumu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obytes%2Fterraform-aws-sumu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obytes%2Fterraform-aws-sumu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/obytes","download_url":"https://codeload.github.com/obytes/terraform-aws-sumu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241244558,"owners_count":19933292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-apigateway","interactive","jwt","lambda","realtime","serverless","sns","sqs","websocket"],"created_at":"2024-11-12T14:10:02.586Z","updated_at":"2025-11-24T11:06:25.488Z","avatar_url":"https://github.com/obytes.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Terraform AWS SUMU\n\n![Websockets](docs/images/sockets.gif)\n\nSUMU(live) is a Generic, Reusable and Pluggable Websocket Stack that can be hooked to any application to make it \ninteractive and provide realtime capability. it consists of the following components:\n\n## Features\n\n- **`Connections Store`**: a DynamoDB where all users active connections are placed, sumu automatically add new\n  connections to the table and delete closed connections from it and prune stale connection using TTL attribute.\n  additionally, it streams **`INSERT`** and **`DELETE`** events, so other apps can track users presence.\n\n- **`Integration Queues/Topics`**: SUMU provides an Input SNS Topic and an Input SQS Queue to receive notifications\n  requests from external applications that want to notify their connected users. It also provides an Output SNS Topic and\n  an Output SQS Queue for external applications to receive messages from connected users.\n\n- **`Websocket Request JWT Authorizer`**: a request JWT Authorizer integrated with connection route, capable of\n  integrating with any JWT IaaS provider (Firebase, Cognito, Auth0...) and able to verify the JWT token signature,\n  expiration time and allowed audiences.\n\n- **`Websockets API Gateway`**: SUMU provides a Websocket API Gateway with connection and disconnection routes\n  integrated with DynamoDB for connections tracking, Keepalive (ping/pong) route to avoid IDLE connections termination\n  and Messages (publish/send) routes integrated with SNS/SQS to fanout messages to external applications.\n\n- **`Websockets Notifications Async Pusher`**: Serverless and Fast AWS API Gateway websockets notifications' pusher\n  using Python AsyncIO for managing asynchronous and concurrent non-blocking IO calls to DynamoDB connections store and\n  API Gateway management API. making it suitable for receiving notifications requests from external applications and\n  broadcasting those messages to multiple users with a fast and cost-effective approach.\n\n- **`Presence Watchdog`**: Connections Tracker for tracking all users connections and notifying external applications\n  about users' presence, It can fanout **`ONLINE`** presence event whenever a user connects and **`OFFLINE`** presence\n  event whenever a user closes all his connections from all devices.\n\n## Deploy It\n\nSUMU can be provisioned with just 2 Terraform modules, the actual sumu module and a helper module to expose the\nwebsocket with custom domain.\n\n```hcl\nmodule \"sumu\" {\n  source      = \"git::https://github.com/obytes/terraform-aws-sumu//modules/serverless\"\n  prefix      = local.prefix\n  common_tags = local.common_tags\n\n  # Authorizer\n  issuer_jwks_uri         = \"https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken@system.gserviceaccount.com\"\n  authorized_audiences    = [\"sumu-websocket\", ]\n  verify_token_expiration = true\n\n  s3_artifacts = {\n     arn    = aws_s3_bucket.artifacts.arn\n     bucket = aws_s3_bucket.artifacts.bucket\n  }\n  github = {\n     owner          = \"obytes\"\n     webhook_secret = \"not-secret\"\n     connection_arn = \"arn:aws:codestar-connections:us-east-1:{ACCOUNT_ID}:connection/{CONNECTION_ID}\"\n  }\n  github_repository               = {\n    authorizer = {\n      name   = \"apigw-jwt-authorizer\"\n      branch = \"main\"\n    }\n    pusher = {\n      name   = \"apigw-websocket-pusher\"\n      branch = \"main\"\n    }\n  }\n  ci_notifications_slack_channels = {\n     info  = \"ci-info\"\n     alert = \"ci-alert\"\n  }\n\n  stage_name      = \"mvp\"\n  apigw_endpoint  = \"https://live.kodhive.com/push\"\n  presence_source = \"queue\"\n}\n\nmodule \"gato\" {\n  source      = \"git::https://github.com/obytes/terraform-aws-gato//modules/core-route53\"\n  prefix      = local.prefix\n  common_tags = local.common_tags\n\n  # DNS\n  r53_zone_id = aws_route53_zone.prerequisite.zone_id\n  cert_arn    = aws_acm_certificate.prerequisite.arn\n  domain_name = \"kodhive.com\"\n  sub_domains = {\n    stateless = \"api\"\n    statefull = \"live\"\n  }\n\n  # Rest APIS\n  http_apis = []\n\n  ws_apis = [\n    {\n      id    = module.sumu.ws_api_id\n      key   = \"live\"\n      stage = module.sumu.ws_api_stage_name\n    }\n  ]\n}\n```\n\n## Usage \n\nSUMU is built to be interoperable with any application, It provides an Output SNS Topic for publishing users messages to \nbackends, an Output SQS Queue for queuing users messages, an Input SNS Topic for external applications to publish \nnotification requests and an Input SQS queue to queue notification requests.\n\n\n### Connecting users\n\n```javascript\nimport Sockette from \"sockette\";\n\nlet endpoint = `wss://live.kodhive.com/push?authorization=${accessToken}`;\nlet ws = new Sockette(endpoint, {\n    onopen: e =\u003e {},\n    onmessage: e =\u003e {},\n    onreconnect: e =\u003e {},\n    onmaximum: e =\u003e {},\n    onclose: e =\u003e {},\n    onerror: e =\u003e {}\n})\n```\n\nTo keep user connections active, you can send ping frames periodically:\n\n```javascript\nimport Sockette from \"sockette\";\nlet keepAliveInterval: any = null;\nfunction keepAliveHandler() {\n    ws.json({action: 'ping'});\n}\nfunction keep_alive() {\n    clearInterval(keepAliveInterval)\n    keepAliveInterval = setInterval(keepAliveHandler, 30000)\n}\nlet endpoint = `wss://live.kodhive.com/push?authorization=${accessToken}`;\nlet ws = new Sockette(endpoint, {\n    onopen: e =\u003e {keep_alive()}\n})\n```\n\n### Sending messages from clients to backend\n\nSUMU is integrated with SNS and SQS, so you can send messages to SNS or publish them to SQS queue, The message should\nbe a JSON String that contains the **`action`** and the actual **`message`**:\n\n- Send a message to backend applications through SQS:\n\n```javascript\nws.json({action: 'send', message: {text: '🦄 Wow so easy!'}});\n```\n\n- Publish a message to backend applications through SNS:\n\n```javascript\nws.json({action: 'publish', message: {text: '🦄 Wow so easy!'}});\n```\n\n### Subscribing backends to clients messages\n\nYou can subscribe a Lambda Function as backend processor of clients messages by creating an SNS subscription and\nallowing SNS to invoke the function\n\n```hcl\nresource \"aws_sns_topic_subscription\" \"_\" {\n  topic_arn = var.messages_topic_arn\n  protocol  = \"lambda\"\n  endpoint  = aws_lambda_function.this.arn\n}\n\nresource \"aws_lambda_permission\" \"with_sns\" {\n  statement_id  = \"AllowExecutionFromSNS\"\n  action        = \"lambda:InvokeFunction\"\n  function_name = aws_lambda_function.this.arn\n  principal     = \"sns.amazonaws.com\"\n  source_arn    = var.messages_topic_arn\n}\n```\n\n\u003e In addition to Lambda, you can publish messages to http webhook endpoints, SMS and Email.\n\n### Polling clients messages from backends\n\nIn case you want to process clients messages in batches you can create an SQS event source and give the Lambda Function\npermission to receive messages from the queue:\n\n```hcl\nresource \"aws_lambda_event_source_mapping\" \"_\" {\n  enabled                            = true\n  batch_size                         = 10\n  event_source_arn                   = var.messages_queue_arn\n  function_name                      = aws_lambda_function.this.arn\n  maximum_batching_window_in_seconds = 0 # Do not wait until batch size is fulfilled\n}\n\ndata \"aws_iam_policy_document\" \"policy\" {\n  statement {\n    actions = [\n      \"sqs:ChangeMessageVisibility\",\n      \"sqs:ChangeMessageVisibilityBatch\",\n      \"sqs:DeleteMessage\",\n      \"sqs:DeleteMessageBatch\",\n      \"sqs:GetQueueAttributes\",\n      \"sqs:ReceiveMessage\"\n    ]\n\n    resources = [\n      var.messages_queue_arn\n    ]\n  }\n}\n```\n\n\u003e SQS is better than SNS if you want to avoid hitting the Lambda Concurrent Execution Limit which is 1,000 (Can be\nincreased to 100,000 by AWS service request)\n\n### Notifying clients from backend\n\nBackend applications can send notifications to AWS API Gateway Websocket connected users by sending a notification\nrequest to the service integrated with the Pusher (SNS|SQS), notifications requests should meet the following format:\n\nFor selective notifications, the message should be a JSON String that contains the list of users and the actual `data`:\n\n```python\nimport json\n\nmessage = {\n    \"users\": [\"783304b1-2320-44db-8f58-09c3035a686b\", \"a280aa41-d99b-4e1c-b126-6f39720633cc\"],\n    \"data\": {\"type\": \"notification\", \"message\": \"A message sent to selected user\"}\n}\nmessage_to_send = json.dumps(message)\n```\n\nFor broadcast notifications, the same but do not provide users list or provide an empty users list:\n\n```python\nimport json\n\nmessage = {\n    \"data\": {\"type\": \"announcement\", \"message\": \"A broadcast to all users\"}\n}\nmessage_to_send = json.dumps(message)\n```\n\n### Notification requests through SNS\n\nSUMU Pusher is subscribing to notifications SNS Topic, and whenever a backend applications Publish notification requests\nto SNS, the later will quickly notify the Pusher by sending the notification request to the subscribed Pusher Lambda.\n\nThis will result in a fast delivery because this approach does not introduce a polling mechanism and SNS will notify the\nPusher whenever a notification request is available. however, at scale SNS will trigger a Pusher Lambda Function for\nevery notification request and given that the Lambda Function Concurrent Invocations Limit is 1,000 per account (Can be\nincreased to 100,000 by support-ticket) notification requests will be throttled for large applications.\n\n\u003e Publish to SNS when you have small application with few users\n\n```python\nimport os\nimport json\nimport time\nimport boto3\n\nmessage = {\n    \"users\": [\"783304b1-2320-44db-8f58-09c3035a686b\", \"a280aa41-d99b-4e1c-b126-6f39720633cc\"],\n    \"data\": {\n        \"type\": \"notification\",\n        \"message\": {\n            \"text\": \"Your order has been fulfilled!\",\n            \"timestamp\": int(time.time())\n        }\n    }\n}\nboto3.client(\"sns\").sns.publish(\n    TargetArn=os.environ[\"NOTIFICATIONS_TOPIC_ARN\"],\n    Message=json.dumps(message),\n)\n```\n\n### Sending notification requests through SQS\n\nThe Pusher can poll notifications from SQS queue,\n\nUnlike SNS, when sending notifications to SQS queue, the Pusher Lambda Function event source is configured to poll\nnotification requests from the SQS Queue, and it will periodically poll notification requests from the Queue using\nPolling Technique.\n\nThis will result in notifications requests to be processed in batches, which comes with many benefits:\n\n- Fewer lambda function executions, to not reach the Lambda Concurrent Execution Limit.\n- As the pusher uses AsyncIO, it will be able to process batches of SQS Records concurrently.\n- Low cost thanks to SQS batches and fewer Lambda Executions.\n\nSUMU meets the same speed and performance of SNS because the SQS queue **`receive_wait_time_seconds`** is set to 0. this\nwill make the Lambda Function do Short Polling instead of Long Polling. and it will receive the notifications requests\nimmediately after being visible on SQS queue.\n\n\u003e Send to SQS when you have a large application with millions of users\n\n```python\nimport os\nimport json\nimport time\nimport boto3\n\nmessage = {\n    \"users\": [\"783304b1-2320-44db-8f58-09c3035a686b\", \"a280aa41-d99b-4e1c-b126-6f39720633cc\"],\n    \"data\": {\n        \"type\": \"notification\",\n        \"message\": {\n            \"text\": \"Your order has been fulfilled!\",\n            \"timestamp\": int(time.time())\n        }\n    }\n}\nboto3.client(\"sqs\").send_message(\n  QueueUrl=os.environ.get(\"NOTIFICATIONS_QUEUE_URL\"),\n  MessageBody=json.dumps(message),\n)\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobytes%2Fterraform-aws-sumu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fobytes%2Fterraform-aws-sumu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobytes%2Fterraform-aws-sumu/lists"}