https://github.com/awsdataarchitect/ecs-fargate-soci
Accelerating LLM Inference on ECS: Leveraging SOCI with AWS Fargate for Lightning-Fast Container Startup
https://github.com/awsdataarchitect/ecs-fargate-soci
Last synced: over 1 year ago
JSON representation
Accelerating LLM Inference on ECS: Leveraging SOCI with AWS Fargate for Lightning-Fast Container Startup
- Host: GitHub
- URL: https://github.com/awsdataarchitect/ecs-fargate-soci
- Owner: awsdataarchitect
- License: apache-2.0
- Created: 2025-03-26T15:10:33.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-26T15:59:21.000Z (over 1 year ago)
- Last Synced: 2025-03-26T16:42:19.609Z (over 1 year ago)
- Language: TypeScript
- Size: 55.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Accelerating LLM Inference on ECS: Leveraging SOCI with AWS Fargate for Lightning-Fast Container Startup
Full AWS-CDK code for Automated SOCI implementation with AWS Fargate and CloudWatch Dashboard showing comparison with non-SOCI.
For more details on how to deploy the infrastructure and the solution details, please refer to the Blog Post:
* [Accelerating LLM Inference on ECS: Leveraging SOCI with AWS Fargate for Lightning-Fast Container Startup)](https://vivek-aws.medium.com/accelerating-llm-inference-on-ecs-leveraging-soci-with-aws-fargate-for-lightning-fast-container-6fb6b7df5b93).
## Useful commands
The `cdk.json` file tells the CDK Toolkit how to execute your app.
* `npm run build` compile typescript to js
* `npm run watch` watch for changes and compile
* `npm run test` perform the jest unit tests
* `npx cdk deploy` deploy this stack to your default AWS account/region
* `npx cdk diff` compare deployed stack with current state
* `npx cdk synth` emits the synthesized CloudFormation template