{"id":19122813,"url":"https://github.com/raohai/aws-lambda-response-streaming","last_synced_at":"2025-06-16T13:10:31.186Z","repository":{"id":232363850,"uuid":"784102892","full_name":"RaoHai/aws-lambda-response-streaming","owner":"RaoHai","description":"An AWS lambda python streaming response example","archived":false,"fork":false,"pushed_at":"2024-04-09T10:54:43.000Z","size":3730,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T00:23:20.101Z","etag":null,"topics":["lambda","openai","serverless","streaming"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RaoHai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-04-09T07:34:11.000Z","updated_at":"2025-03-07T13:53:08.000Z","dependencies_parsed_at":"2024-04-09T11:52:42.774Z","dependency_job_id":"67add6f3-6a65-42cd-9948-046df18ef5c1","html_url":"https://github.com/RaoHai/aws-lambda-response-streaming","commit_stats":null,"previous_names":["raohai/aws-lambda-response-streaming"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RaoHai/aws-lambda-response-streaming","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RaoHai%2Faws-lambda-response-streaming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RaoHai%2Faws-lambda-response-streaming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RaoHai%2Faws-lambda-response-streaming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RaoHai%2Faws-lambda-response-streaming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RaoHai","download_url":"https://codeload.github.com/RaoHai/aws-lambda-response-streaming/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RaoHai%2Faws-lambda-response-streaming/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260166308,"owners_count":22968635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lambda","openai","serverless","streaming"],"created_at":"2024-11-09T05:23:08.758Z","updated_at":"2025-06-16T13:10:31.151Z","avatar_url":"https://github.com/RaoHai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Building streaming functions with Python on AWS Lambda\n\nThis example shows streaming response from OpenAI completions with FastAPI on AWS Lambda.\n\nCredit to [aws-lambda-web-adapter](https://github.com/awslabs/aws-lambda-web-adapter)\n\n![Architecture](imgs/architecture.png)\n\n## How does it work?\n\nThis example uses FastAPI provides inference API. The inference API endpoint invokes OpenAI, and streams the response. Both Lambda Web Adapter and function URL have response streaming mode enabled. So the response from OpenAI are streamed all the way back to the client. \n\nThis function is packaged as a Docker image. Here is the content of the Dockerfile. \n\n```dockerfile\nFROM public.ecr.aws/docker/library/python:3.12.0-slim-bullseye\n\nCOPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.8.1 /lambda-adapter /opt/extensions/lambda-adapter\n\n# Copy function code\nCOPY . ${LAMBDA_TASK_ROOT}\n# from your project folder.\nCOPY requirements.txt .\nRUN pip3 install -r requirements.txt --target \"${LAMBDA_TASK_ROOT}\" -U --no-cache-dir\n\nCMD [\"python\", \"main.py\"]\n```\n\nNotice that we only need to add the second line to install Lambda Web Adapter. \n\n```dockerfile\nCOPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.8.1 /lambda-adapter /opt/extensions/\n```\n\nIn the SAM template, we use an environment variable `AWS_LWA_INVOKE_MODE: RESPONSE_STREAM` to configure Lambda Web Adapter in response streaming mode. And adding a function url with `InvokeMode: RESPONSE_STREAM`. \n\n```yaml\n  FastAPIFunction:\n    Type: AWS::Serverless::Function\n    Properties:\n      PackageType: Image\n      MemorySize: 512\n      Environment:\n        Variables:\n          AWS_LWA_INVOKE_MODE: RESPONSE_STREAM\n      FunctionUrlConfig:\n        AuthType: NONE\n        InvokeMode: RESPONSE_STREAM\n      Policies:\n      - Statement:\n        - Sid: BedrockInvokePolicy\n          Effect: Allow\n          Action:\n          - bedrock:InvokeModelWithResponseStream\n          Resource: '*'\n```      \n\n## Build and deploy\n\nRun the following commends to build and deploy this example. \n\n```bash\nsam build --use-container\nsam deploy --guided\n```\n\n## Test the example\n\nAfter the deployment completes, curl the `FastAPIFunctionUrl`.\n\n```bash\ncurl -v -N --location '${{FastAPIFunctionUrl}}/api/chat/stream' \\\n--header 'Content-Type: application/json' \\\n--header 'Transfer-Encoding: chunked' \\\n--data '{\"messages\":[{\"role\":\"user\",\"content\":\"Count to 100, with a comma between each number and no newlines. E.g., 1, 2, 3, ...\"}],\"prompt\":\"\"}'\n```\n\n\n![Demo](imgs/demo.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraohai%2Faws-lambda-response-streaming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fraohai%2Faws-lambda-response-streaming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fraohai%2Faws-lambda-response-streaming/lists"}