https://github.com/samshad/csci-5410-serverless-data-processing
CSCI 5410 – Serverless Data Processing (Dalhousie, Summer 2024). AWS & GCP labs plus a capstone React + Lambda feedback platform showcasing event‑driven pipelines, Step Functions orchestration, and DynamoDB persistence 100 % serverless.
https://github.com/samshad/csci-5410-serverless-data-processing
aws aws-lambda cloud cloudwatch-logs cognito data-processing dynamodb event-driven-architecture gcp-cloud-functions iac javascript python react s3 serverless step-functions
Last synced: about 1 month ago
JSON representation
CSCI 5410 – Serverless Data Processing (Dalhousie, Summer 2024). AWS & GCP labs plus a capstone React + Lambda feedback platform showcasing event‑driven pipelines, Step Functions orchestration, and DynamoDB persistence 100 % serverless.
- Host: GitHub
- URL: https://github.com/samshad/csci-5410-serverless-data-processing
- Owner: samshad
- License: mit
- Created: 2024-06-07T00:59:29.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2025-05-13T23:05:05.000Z (about 1 year ago)
- Last Synced: 2025-05-21T02:44:53.612Z (about 1 year ago)
- Topics: aws, aws-lambda, cloud, cloudwatch-logs, cognito, data-processing, dynamodb, event-driven-architecture, gcp-cloud-functions, iac, javascript, python, react, s3, serverless, step-functions
- Language: JavaScript
- Homepage:
- Size: 37.3 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# CSCI 5410 — Serverless Data Processing
Dalhousie University · Summer 2024
Author · Md Samshad Rahman (`@samshad`)
> A bite‑sized journey through **serverless architectures** from AWS Lambda image pipelines and Google Cloud Functions to a full‑stack feedback‑analytics platform orchestrated entirely with managed services.
---
## 🌐 What you’ll find here
| Folder | Theme | Key tech & skills |
|--------|-------|-------------------|
| **Lab/Lab1** | *Image‑processing on AWS Lambda* | Python 3.11 · Boto3 · S3 · DynamoDB
Upload, resize & audit images |
| **Lab/Lab2** | *GCP Cloud Functions & Pub/Sub* | Python · Google Cloud Functions · Pub/Sub triggers; serverless sentiment extraction using Google Cloud NLP API |
| **Lab/Lab3** | *Event‑driven data pipeline* | AWS Step Functions · SQS · SNS; orchestrates multi‑stage ETL (code in `Lab/Lab3/`) |
| **Project** | *Serverless feedback & room‑booking platform* | React front‑end + Python Lambda back‑end:
• **Backend** – 23 Lambda functions for auth, CRUD, sentiment analysis, chat‑bot helpers, and DynamoDB persistence volume
• **Frontend** – Create React App dashboard & forms for admins and users |
> ⭐ **Languages:** ≈ 51 % JavaScript, 31 % Python, 16 % CSS, 2 % HTML
> 📁 **Repo activity:** 16 commits, labs + capstone milestones
---
## 🧠 Learning outcomes
* **Serverless primitives** – authoring & provisioning Lambda, Cloud Functions, and Step Functions without servers to patch.
* **Event‑driven design** – S3/Object events, Pub/Sub and API Gateway triggers; fan‑in/out patterns with queues & topics.
* **Poly‑cloud mindset** – mixed AWS + GCP toolchains; credential isolation and cross‑platform SDK use.
* **Data processing pipelines** – on‑the‑fly image optimisation, feedback ingestion, sentiment scoring, and dashboard aggregation.
* **Full‑stack delivery** – React SPA backed by RESTful Lambda APIs & DynamoDB—CI‑friendly and cost‑efficient.
---
## 🛠 Toolchain
| Domain | Stack |
|--------|-------|
| **Back‑end** | AWS Lambda · API Gateway · DynamoDB · S3 · Step Functions · GCP Cloud Functions · Cloud NLP |
| **Front‑end** | React 18 · Vite/CRA · Tailwind CSS |
| **Infra‑as‑Code** | AWS SAM · CloudFormation & `gcloud` CLI scripts |
| **Auth** | Amazon Cognito |
| **DevOps** | GitHub Actions (lint + deploy), pre‑commit hooks |
---
## 📂 Repository layout
```
.
├── Lab/
│ ├── Lab1/ # Image optimise pipeline (AWS)
│ ├── Lab2/ # GCP Cloud Functions + Pub/Sub
│ └── Lab3/ # Serverless ETL with Step Functions
├── Project/
│ ├── backend/ # Python Lambda functions & utils
│ └── frontend/ # React SPA
└── .gitignore
```
Each sub‑folder ships a mini‑README or docstrings with deployment steps.
---
## 📄 Licence
This project is licensed under the MIT License.
See the [LICENSE](LICENSE) file for details.
© 2025 Md Samshad Rahman