{"id":19910844,"url":"https://github.com/fun-with-serverless/rustifying-serverless","last_synced_at":"2025-05-03T03:30:41.803Z","repository":{"id":191590613,"uuid":"684974156","full_name":"fun-with-serverless/rustifying-serverless","owner":"fun-with-serverless","description":"How to use Rust in your existing Serverless projects","archived":false,"fork":false,"pushed_at":"2024-02-13T08:22:45.000Z","size":1088,"stargazers_count":45,"open_issues_count":0,"forks_count":3,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-07T10:46:05.279Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fun-with-serverless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-30T08:37:02.000Z","updated_at":"2025-03-03T23:34:43.000Z","dependencies_parsed_at":null,"dependency_job_id":"7d465ade-0d6d-4d43-b178-7d3bbceb085e","html_url":"https://github.com/fun-with-serverless/rustifying-serverless","commit_stats":null,"previous_names":["fun-with-serverless/rustifying-serverless"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fun-with-serverless%2Frustifying-serverless","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fun-with-serverless%2Frustifying-serverless/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fun-with-serverless%2Frustifying-serverless/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fun-with-serverless%2Frustifying-serverless/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fun-with-serverless","download_url":"https://codeload.github.com/fun-with-serverless/rustifying-serverless/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252137501,"owners_count":21700224,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T21:21:34.523Z","updated_at":"2025-05-03T03:30:40.753Z","avatar_url":"https://github.com/fun-with-serverless.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# “Rustifying” serverless: Boost AWS Lambda performance with Rust (COM306)\n\n## Introduction\nThis repository serves as a companion to my talk at AWS ReInvent. The talk, titled \"“Rustifying” serverless: Boost AWS Lambda performance with Rust\", explores how to leverage the performance advantages of Rust in serverless AWS Lambda functions. For more details and to watch the talk, visit [ReInvent Talk](https://www.youtube.com/watch?v=Mdh_2PXe9i8).\n\n## Background Story\n### The Journey\n- As a company focused on developing AWS management tools, we started with a Minimum Viable Product (MVP) for S3 bucket management.\n- Initially, the product was open only to our most loyal customers, and we received overwhelmingly positive feedback.\n- Eventually, we opened the product to all of our users.\n- At its peak, the service catered to thousands of clients and tens of thousands of users.\n  \n### Challenges\n- We started experiencing performance bottlenecks and increased IT spending.\n- Major architectural changes like adding cache and improving algorithms were implemented.\n\n### The Decision\n- We reached a point where our existing runtime, Python, was not sufficient for our performance needs.\n- Thus, we decided to leverage Rust for its performance advantages.\n\nThis repository illustrates the journey that led us to this point.\n\n## Architecture\n![Architecture Diagram](https://github.com/fun-with-serverless/rustifying-serverless/assets/110536677/7950f8d2-1e80-4681-8938-339ef6b203f6)\n\n- The architecture consists of an API Gateway with a Lambda authorizer.\n- Multiple Lambda functions are connected to the API Gateway. These functions interact with services like S3 and DynamoDB.\n- We have multiple runtimes, mostly Python, but also NodeJS.\n\n## Ways to Integrate Rust into Your Serverless Workload\nThere are primarily three ways to integrate Rust:\n\n### 1. Rust Bindings\n#### Python\n- Use Pyo3 with maturin to create a `.whl` package for use in your Python Lambda.\n- Example code can be found under [s3-ops-rust-pyo3-lib](./s3-ops-rust-pyo3-lib).\n\n#### NodeJS\n- Use NAPI-RS to package and build your code. A one stop CLI, unlike Python where you need to use teo tools.\n- Examples can be found under [s3-ops-rust-napi-lib](./s3-ops-rust-napi-lib).\n\n**Benefits**: Speed of development and no need to rewrite the Lambda function.\n\n### 2. Rewrite the Lambda\n- Use `cargo-lambda` and AWS SAM to deploy a full-fledged Rust Lambda.\n- Example code can be found under [s3-admin-app/authorizer_rust](./s3-admin-app/authorizer_rust).\n- **Benefits**: Reduced cold starts, thanks to the Rust runtime.\n\n### 3. Use Extensions\n- Use extensions to address cross-cutting concerns, such as analytics reporting, using a fast language like Rust.\n- Example code can be found under [analytics-extension](./analytics-extension).\n- **Benefits**: Reduce Lambda latency by offloading tasks to an external process.\n\n## Building the Example\n- This example comes with a devcontainer for both Rust and Python development.\n- In case you are not using DevContainers, make sure you have the following prerequisites:\n  - Python 3.11\n  - Rust - latest version\n  - Poetry\n  - Poe \n  - AWS SAM\n  - NodeJS 20\n  - AWS CLI\n  - NAPI-RS\n- Prepare the NodeJS Rust Library by running:\n```\ncd s3-ops-rust-napi-lib\nnpm i\n``` \n- We use `poetry` for build management. To build and deploy, run:\n  ```bash\n  # Install dev tools used for rust compilation.\n  poetry install --only=rust-dev-tools\n  # Build the rust package\n  poe build-python-lib\n  poe build-node-lib\n  # Build and deploy the extension\n  poe build-and-deploy-extension\n  ```\n- Update [s3-asmin-app/template.yaml](./s3-admin-app/template.yaml) with the ARN of the extension under `Globals/Layers`.\n- Build the application and deploy it - `poe build-and-deploy-app`\n- To create a new user in the 'users' table, follow the AWS SAM output. For example `aws dynamodb put-item --table-name ...`\n- Make API calls to the various APIs as defined in the AWS SAM output.\n\n## Local development\nThe main application resides in `s3-admin-app`,  Rust bindings are in `s3-ops-rust-lib` and the extension in `analytics-extension`.\n ```bash\n  # Install dev tools used for rust compilation.\n  poetry install --only=rust-dev-tools\n  # Build the rust package\n  poe build-python-lib\n  poe build-node-lib\n  # Add the local rust package\n  poetry add .rust-lib/s3_ops_rust-0.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl --group dev \n  poetry install\n  ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffun-with-serverless%2Frustifying-serverless","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffun-with-serverless%2Frustifying-serverless","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffun-with-serverless%2Frustifying-serverless/lists"}