{"id":31908306,"url":"https://github.com/vishapp/scalable_serverless_image_upload","last_synced_at":"2025-10-13T15:26:57.127Z","repository":{"id":316261464,"uuid":"1062659029","full_name":"VishApp/scalable_serverless_image_upload","owner":"VishApp","description":"A scalable serverless image upload, storage, and retrieval service built with FastAPI, AWS services, and LocalStack for local development","archived":false,"fork":false,"pushed_at":"2025-09-23T14:53:01.000Z","size":32,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-23T16:32:31.249Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VishApp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-23T14:47:51.000Z","updated_at":"2025-09-23T14:53:05.000Z","dependencies_parsed_at":"2025-09-23T16:32:33.716Z","dependency_job_id":null,"html_url":"https://github.com/VishApp/scalable_serverless_image_upload","commit_stats":null,"previous_names":["vishapp/scalable_serverless_image_upload"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/VishApp/scalable_serverless_image_upload","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VishApp%2Fscalable_serverless_image_upload","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VishApp%2Fscalable_serverless_image_upload/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VishApp%2Fscalable_serverless_image_upload/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VishApp%2Fscalable_serverless_image_upload/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VishApp","download_url":"https://codeload.github.com/VishApp/scalable_serverless_image_upload/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VishApp%2Fscalable_serverless_image_upload/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279015918,"owners_count":26085778,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-13T15:26:51.030Z","updated_at":"2025-10-13T15:26:57.120Z","avatar_url":"https://github.com/VishApp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scalable Serverless Image Upload Service\n\nA scalable serverless image upload, storage, and retrieval service built with FastAPI, AWS services, and LocalStack for local development.\n\n## 🚀 Features\n\n- **Image Upload**: Support for JPEG, PNG, GIF, WebP formats with metadata\n- **Image Listing**: Paginated listing with filtering by user, tags, and date\n- **Image Retrieval**: Get individual images with presigned download URLs\n- **Image Deletion**: Soft delete with user authorization\n- **Scalable Architecture**: Built on AWS serverless services\n- **Local Development**: Complete LocalStack setup for offline development\n- **Comprehensive Testing**: Unit tests with high coverage\n- **API Documentation**: Interactive Swagger/ReDoc documentation\n\n## 🏗️ Architecture\n\n- **API Gateway**: HTTP API entry point\n- **Lambda Functions**: Serverless compute with FastAPI\n- **S3**: Object storage for images\n- **DynamoDB**: NoSQL database for metadata with GSIs for efficient querying\n- **LocalStack**: Local AWS service emulation\n\n## 📋 Prerequisites\n\n- Python 3.7+\n- Docker and Docker Compose\n- AWS CLI (for LocalStack)\n\n## 🛠️ Quick Start\n\n### 1. Start the project\n\n```bash\ncd infrastructure\ndocker-compose up -d\n```\n\n### 2. Access the API\n\n- **API Base URL**: http://localhost:8000\n- **Interactive Docs**: http://localhost:8000/docs\n- **ReDoc**: http://localhost:8000/redoc\n- **Health Check**: http://localhost:8000/health\n\n## 📝 Usage Examples\n\n### Upload an Image\n\n```bash\ncurl -X POST \"http://localhost:8000/images\" \\\n  -H \"Content-Type: multipart/form-data\" \\\n  -F \"file=@path/to/image.jpg\" \\\n  -F \"user_id=user123\" \\\n  -F \"title=Beautiful Sunset\" \\\n  -F \"description=A stunning sunset over the mountains\" \\\n  -F \"tags=nature,sunset,mountains\"\n```\n\n### List Images\n\n```bash\n# List all images\ncurl \"http://localhost:8000/images\"\n\n# List with filters\ncurl \"http://localhost:8000/images?user_id=user123\u0026tags=nature\u0026limit=10\"\n\n# List user images\ncurl \"http://localhost:8000/users/user123/images\"\n\n# List by tag\ncurl \"http://localhost:8000/tags/nature/images\"\n```\n\n### Get Image Details\n\n```bash\ncurl \"http://localhost:8000/tags/nature/images\"\n```\n\n### Get Download URL\n\n```bash\ncurl \"http://localhost:8000/images/{image_id}/download\"\n```\n\n### Delete Image\n\n```bash\ncurl -X DELETE \"http://localhost:8000/images/{image_id}\" \\\n  -H \"X-User-Id: user123\"\n```\n\n## 🧪 Running Tests\n\n### Run All Tests\n\n```bash\npytest\n```\n\n### Run with Coverage\n\n```bash\npytest --cov=src --cov-report=html\n```\n\n### Run Specific Test Categories\n\n```bash\n# Unit tests only\npytest tests/test_utils/ tests/test_services/\n\n# Integration tests\npytest tests/test_handlers/\n\n# Specific test file\npytest tests/test_services/test_image_service.py -v\n```\n\n## 🏃‍♂️ Development Workflow\n\n### 1. Start All Services\n\n```bash\ndocker-compose up -d\n```\n\n### 2. Set Environment Variables\n\n```bash\nexport AWS_ACCESS_KEY_ID=test\nexport AWS_SECRET_ACCESS_KEY=test\nexport AWS_DEFAULT_REGION=us-east-1\nexport AWS_ENDPOINT_URL=http://localhost:4566\nexport S3_BUCKET_NAME=instagram-images-dev\nexport DYNAMODB_TABLE_NAME=ImageMetadata-dev\n```\n\n### 3. Development Server\n\n```bash\nuvicorn src.main:app --reload --host 0.0.0.0 --port 8000\n```\n\n### 4. Code Quality\n\n```bash\n# Format code\nblack src/ tests/\n\n# Lint code\nflake8 src/ tests/\n\n# Type checking\nmypy src/\n```\n\n## 📊 Project Structure\n\n```\nscalable_serverless_image_upload/\n├── src/\n│   ├── handlers/           # Individual Lambda handlers\n│   ├── services/          # Business logic\n│   ├── models/            # Pydantic data models\n│   ├── utils/             # Utility classes\n│   └── main.py            # Main FastAPI application\n├── tests/                 # Comprehensive test suite\n├── infrastructure/        # LocalStack and AWS setup\n├── docs/                  # API documentation\n├── requirements.txt       # Python dependencies\n├── pytest.ini           # Test configuration\n└── README.md             # This file\n```\n\n## 🔧 Configuration\n\n### Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `AWS_ACCESS_KEY_ID` | `test` | AWS access key (LocalStack) |\n| `AWS_SECRET_ACCESS_KEY` | `test` | AWS secret key (LocalStack) |\n| `AWS_DEFAULT_REGION` | `us-east-1` | AWS region |\n| `AWS_ENDPOINT_URL` | `http://localhost:4566` | LocalStack endpoint |\n| `S3_BUCKET_NAME` | `instagram-images-dev` | S3 bucket name |\n| `DYNAMODB_TABLE_NAME` | `ImageMetadata-dev` | DynamoDB table name |\n\n### File Constraints\n\n- **Formats**: JPEG, PNG, GIF, WebP\n- **Size**: 1KB - 10MB\n- **Dimensions**: 50x50 - 4000x4000 pixels\n- **Metadata**: Title (200 chars), Description (1000 chars), Tags (10 max, 50 chars each)\n\n## 🚀 Deployment\n\n### AWS Lambda Deployment\n\n1. **Package the application:**\n```bash\npip install -t package/ -r requirements.txt\ncp -r src/ package/\ncd package \u0026\u0026 zip -r ../deployment.zip . \u0026\u0026 cd ..\n```\n\n2. **Deploy with AWS CLI:**\n```bash\naws lambda create-function \\\n  --function-name instagram-image-service \\\n  --runtime python3.9 \\\n  --role arn:aws:iam::account:role/lambda-role \\\n  --handler src.main.handler \\\n  --zip-file fileb://deployment.zip\n```\n\n3. **Configure API Gateway:**\n- Create HTTP API\n- Add Lambda integration\n- Configure routes: `ANY /{proxy+}`\n\n### Serverless Framework (Alternative)\n\n```bash\nnpm install -g serverless\nsls deploy\n```\n\n## 🔍 Monitoring and Observability\n\n### Local Development\n\n- **Logs**: Application logs via Python logging\n- **Metrics**: Basic FastAPI metrics\n- **Health**: `/health` endpoint\n\n### Production Recommendations\n\n- **AWS CloudWatch**: Lambda logs and metrics\n- **AWS X-Ray**: Distributed tracing\n- **Custom Metrics**: Business metrics via CloudWatch\n- **Alarms**: Error rate and latency monitoring\n\n## 🔒 Security Considerations\n\n### Current Implementation\n\n- Input validation and sanitization\n- File type and size restrictions\n- User authorization for delete operations\n- Presigned URLs for secure downloads\n\n### Production Enhancements\n\n- **Authentication**: JWT or OAuth integration\n- **Rate Limiting**: Per-user and IP-based limits\n- **CORS**: Proper origin configuration\n- **Encryption**: S3 encryption at rest\n- **WAF**: Web Application Firewall\n- **Secrets**: AWS Secrets Manager for credentials\n\n\n## 📚 API Documentation\n\nDetailed API documentation is available at:\n- **Interactive Docs**: http://localhost:8000/docs\n- **Static Docs**: [docs/api-documentation.md](docs/api-documentation.md)\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Add tests for new functionality\n4. Ensure all tests pass\n5. Submit a pull request\n\n## 📄 License\n\nThis project is licensed under the MIT License.\n\n## 🆘 Support\n\nFor issues and questions:\n1. Check the troubleshooting section\n2. Review existing issues\n3. Create a new issue with detailed information","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvishapp%2Fscalable_serverless_image_upload","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvishapp%2Fscalable_serverless_image_upload","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvishapp%2Fscalable_serverless_image_upload/lists"}