{"id":25220349,"url":"https://github.com/dvarshith/aws-face-recognition","last_synced_at":"2025-12-30T23:07:53.572Z","repository":{"id":276394393,"uuid":"929140902","full_name":"dvarshith/aws-face-recognition","owner":"dvarshith","description":"Serverless face recognition pipeline on AWS (EC2, Lambda, S3, SQS, auto-scaling).","archived":false,"fork":false,"pushed_at":"2025-02-07T23:57:05.000Z","size":408,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-08T00:27:02.190Z","etag":null,"topics":["aws","cloud-computing","ec2","face-recognition","java","lambda","python","s3","serverless","sqs"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dvarshith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-07T22:05:43.000Z","updated_at":"2025-02-07T23:57:08.000Z","dependencies_parsed_at":"2025-02-08T00:37:09.541Z","dependency_job_id":null,"html_url":"https://github.com/dvarshith/aws-face-recognition","commit_stats":null,"previous_names":["dvarshith/aws-face-recognition"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvarshith%2Faws-face-recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvarshith%2Faws-face-recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvarshith%2Faws-face-recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvarshith%2Faws-face-recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dvarshith","download_url":"https://codeload.github.com/dvarshith/aws-face-recognition/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247326573,"owners_count":20920869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloud-computing","ec2","face-recognition","java","lambda","python","s3","serverless","sqs"],"created_at":"2025-02-10T21:52:33.627Z","updated_at":"2025-12-30T23:07:53.566Z","avatar_url":"https://github.com/dvarshith.png","language":"Jupyter Notebook","readme":"# AWS Face Recognition (Serverless \u0026 Auto-scaling)\n\n[![AWS Services](https://img.shields.io/badge/AWS-EC2%2C%20S3%2C%20Lambda%2C%20SQS-orange)](https://aws.amazon.com/)\n[![Java 17](https://img.shields.io/badge/Java-17-blue.svg)](https://www.java.com/)\n[![Python 3.8+](https://img.shields.io/badge/Python-3.8+-blue.svg)](https://www.python.org/)\n\n \u003c/br\u003e\nA comprehensive AWS-based face recognition system that demonstrates:\n\n- **Video Splitting** with AWS Lambda (using FFmpeg)\n- **Face Detection \u0026 Recognition** with a pre-trained CNN (ResNet)  \n- **Auto-scaling** on EC2 based on SQS message queue depth\n- **Serverless Pipelines** (S3 triggers, asynchronous Lambda invocations)\n\n\n\u003e **Note**: This project contains subfolders for different parts (Project1, Project2, Project3).\n\n\n\n \u003c/br\u003e\n \n## Table of Contents\n\n1. [Overview](#overview)  \n2. [Repository Structure](#repository-structure)  \n3. [Architecture](#architecture)  \n4. [Prerequisites](#prerequisites)  \n5. [Setup \u0026 Installation](#setup--installation)  \n6. [Usage](#usage)  \n7. [Security Considerations](#security-considerations)  \n8. [Possible Improvements](#possible-improvements)  \n9. [Acknowledgments](#acknowledgments)  \n10. [License](#license)  \n11. [Contact](#contact)\n\n\n\n \u003c/br\u003e\n \n## Overview\n\nThis repository showcases an **end-to-end face recognition pipeline** using AWS:\n- **Input Bucket**: Users upload `.mp4` videos to `\u003cASU_ID\u003e-input`.\n- **Video-Splitting Lambda**: Extracts frames from uploaded videos and stores them in `\u003cASU_ID\u003e-stage-1`.\n- **Face-Recognition Lambda**: Detects faces, classifies them, and writes the result to `\u003cASU_ID\u003e-output`.\n- **Auto-scaling App Tier**: Launches or terminates EC2 instances based on SQS queue length.\n- **Web Tier**: A Spring Boot REST API that interacts with SQS and S3.\n\nOriginally developed for a Cloud Computing course, but it’s a solid reference for real-world AWS patterns.\n\n\n\n\n \u003c/br\u003e\n \n## Repository Structure\n```\naws-face-recognition/\n├── Project1/                        # Basic AWS resource management (EC2, S3, SQS)\n├── Project2/                        # IaaS-based face recognition with auto-scaling\n│   ├── src/\n│   ├── pom.xml\n│   └── ...\n├── Project3/                        # PaaS-based (Lambda) video splitting \u0026 face recognition\n│   ├── video-splitting/\n│   ├── face-recognition/\n│   └── ...\n├── .gitignore\n├── README.md                        # You're here!\n└── Other scripts/data as needed\n```\n\n\n \u003c/br\u003e\n \n## Architecture\n\nA high-level flow: \u003c/br\u003e\nUser (video) -\u003e [S3: \u003cASU_ID\u003e-input] -\u003e [Lambda: video-splitting] -\u003e [S3: \u003cASU_ID\u003e-stage-1] -\u003e [Lambda: face-recognition] -\u003e [S3: \u003cASU_ID\u003e-output]\n\nFor the IaaS version (Project 2): \u003c/br\u003e\nUser -\u003e Web Tier (Spring Boot) -\u003e SQS -\u003e [Auto-Scaling EC2 App Tier] -\u003e S3\n\nEach part uses AWS components:\n- **EC2** for the App Tier.\n- **S3** for input/output storage.\n- **SQS** for message queue handling.\n- **Lambda** for serverless tasks (video splitting \u0026 face recognition).\n- **IAM** for access control (best done with roles rather than keys in code).\n\n\n\n\n \u003c/br\u003e\n \n## Prerequisites\n\n1. **AWS Account**  \n   - S3, EC2, Lambda, SQS, IAM permissions.\n2. **Java 17** \u0026 **Maven**  \n   - For running Spring Boot or other Java-based code.\n3. **Python 3.8+**  \n   - Required for the face-recognition scripts, ffmpeg steps, etc.\n4. **Docker**  \n   - For building container images for Lambda.\n\n\n\n\n \u003c/br\u003e\n \n## Setup \u0026 Installation\n\n1. **Clone this Repository**\n   ```\n   git clone [https://github.com/\u003cYourUsername\u003e/aws-face-recognition.git](https://github.com/dvarshith/aws-face-recognition.git)\n   cd aws-face-recognition\n   ```\n\n2. **Configure AWS Credentials**\n   - **Recommended**: Use IAM Roles for EC2 and Lambda so you don’t store keys in code.\n   - Or set environment variables locally (for testing):\n     ```\n     export AWS_ACCESS_KEY_ID=\u003cYourAccessKey\u003e\n     export AWS_SECRET_ACCESS_KEY=\u003cYourSecretKey\u003e\n     export AWS_DEFAULT_REGION=us-east-1\n     ```\n\n3. **Build the Java Projects**\n   ```\n   cd Project2\n   mvn clean package\n   ```\n\n4. Deploy the Lambdas\n   - For video-splitting and face-recognition (Project 3), either:\n     - Upload the JARs to AWS Lambda via console, _or_\n     - Build Docker images with their respective Dockerfile and push to ECR, then create Lambdas from those images.\n\n\n\n\n \u003c/br\u003e\n \n## Usage\n1. Upload a Video (PaaS Example)\n   - Upload test_00.mp4 to \u003cASU_ID\u003e-input.\n   - Lambda (video-splitting) extracts one frame, saves as test_00.jpg in \u003cASU_ID\u003e-stage-1.\n   - Lambda (face-recognition) runs face detection, saves the recognized name in test_00.txt in \u003cASU_ID\u003e-output.\n2. Web Tier (IaaS Example)\n   - Run:\n      ```\n      java -jar target/Project2-0.0.1-SNAPSHOT.jar\n      ```\n   - Upload: POST / with form-data:\n      ```\n      inputFile=\u003cimage_or_video_file\u003e\n      ```\n   - The API sends a message to SQS, triggers the app tier, and eventually returns the classification result.\n\n\n\n \u003c/br\u003e\n\n## Security Considerations\n- Remove Hardcoded Credentials: Do not store AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY in Java/Python files.\n- Use IAM Roles: Prefer instance profiles for EC2, and execution roles for Lambda.\n- .gitignore: Make sure you aren’t committing .pem files, .log files with private data, or large training data sets.\n\n\n\n \u003c/br\u003e\n\n## Possible Improvements\n- Add CI/CD (GitHub Actions or Jenkins) for automatic builds and deployments.\n- More Granular IAM Policies to adhere to the principle of least privilege.\n- Performance Tuning for your Lambda containers (memory, concurrency).\n- CloudWatch Alarms \u0026 Metrics for deeper monitoring of queue length, CPU, memory usage, etc.\n\n\n\n \u003c/br\u003e\n\n## Acknowledgments\n- Dataset, test cases, etc. provided by Vista Lab from Arizona State University [https://github.com/visa-lab/CSE546-Cloud-Computing/tree/main].\n\n\n\n \u003c/br\u003e\n\n## License\nThis project is released under the `MIT License`. That means you’re free to use, modify, and distribute the code, but you do so at your own risk.\n\n\n\n \u003c/br\u003e\n\n## Contact\nAuthor: Varshith Dupati \u003c/br\u003e\nGitHub: @dvarshith \u003c/br\u003e\nEmail: dvarshith942@gmail.com \u003c/br\u003e\nIssues: Please open an issue on this repo if you have questions or find bugs. \u003c/br\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvarshith%2Faws-face-recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdvarshith%2Faws-face-recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvarshith%2Faws-face-recognition/lists"}