https://github.com/victorcezeh/aws-event-driven-data-platform-project
A production-style serverless event-driven data platform that ingests semi-structured JSON data from an external API, transforms and validates it using AWS Lambda inside a private VPC, and loads curated datasets into a Redshift data warehouse. Fully provisioned with CloudFormation and deployed via GitHub Actions CI/CD.
https://github.com/victorcezeh/aws-event-driven-data-platform-project
aws-lambda aws-s3 cloudformation git github-actions python redshift vpc yaml
Last synced: 22 days ago
JSON representation
A production-style serverless event-driven data platform that ingests semi-structured JSON data from an external API, transforms and validates it using AWS Lambda inside a private VPC, and loads curated datasets into a Redshift data warehouse. Fully provisioned with CloudFormation and deployed via GitHub Actions CI/CD.
- Host: GitHub
- URL: https://github.com/victorcezeh/aws-event-driven-data-platform-project
- Owner: victorcezeh
- Created: 2026-01-17T03:36:09.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-06-02T05:52:30.000Z (24 days ago)
- Last Synced: 2026-06-02T07:24:13.655Z (24 days ago)
- Topics: aws-lambda, aws-s3, cloudformation, git, github-actions, python, redshift, vpc, yaml
- Language: Python
- Homepage:
- Size: 23.5 MB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# aws-event-driven-data-platform 🏗️
> 🚧 In progress - built alongside AWS SAA exam prep 😁
## Context
This project started as a way to make my AWS SAA studying concrete. Instead of just reading about S3, VPC, IAM, and CloudFormation, I wanted to actually build something with them. Something that resembles a real production data platform.
The result is an event-driven pipeline that ingests raw JSON from an external API, processes it with AWS Lambda inside a private VPC, and loads the results into a Redshift Serverless data warehouse. Fully provisioned with CloudFormation.
## Infrastructure 🔧
Each stack is purpose-built and independently deployable via `main_stack.yaml`.
| Stack | What it does |
|-------|-------------|
| `vpc.yaml` | 🌐 Private VPC, subnets, route tables, VPC endpoints |
| `iam_role.yaml` | 🔐 Least-privilege IAM roles for Lambda and Redshift |
| `s3.yaml` | 🪣 S3 bucket with versioning, HTTPS enforcement, event trigger |
| `secrets_manager.yaml` | 🔑 Auto-generated Redshift credentials |
| `lambda.yaml` | ⚡ Python 3.14 Lambda inside private VPC |
| `redshift.yaml` | 🏭 Redshift Serverless in private subnet |
| `monitoring.yaml` | 📊 CloudWatch alarms, log groups, SNS email alerts |
| `main_stack.yaml` | 🎯 Nested stack orchestrator |