{"id":50522618,"url":"https://github.com/lyushher/file-upload-service","last_synced_at":"2026-06-03T05:31:12.537Z","repository":{"id":361274994,"uuid":"1037592551","full_name":"lyushher/file-upload-service","owner":"lyushher","description":"Backend system that enables secure and efficient browser-based file uploads to cloud storage without exposing server credentials.","archived":false,"fork":false,"pushed_at":"2026-05-29T21:35:34.000Z","size":46,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-29T23:14:12.114Z","etag":null,"topics":["api","aws-s3","backend","cloud","cloud-storage","devops","docker","fastapi","file-upload","postgresql","presigned-url","python-backend","rest-api","s3-presigned-url","secure-upload"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lyushher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-13T20:12:41.000Z","updated_at":"2026-05-29T21:35:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lyushher/file-upload-service","commit_stats":null,"previous_names":["lyushher/file-upload-service"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/lyushher/file-upload-service","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyushher%2Ffile-upload-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyushher%2Ffile-upload-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyushher%2Ffile-upload-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyushher%2Ffile-upload-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lyushher","download_url":"https://codeload.github.com/lyushher/file-upload-service/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyushher%2Ffile-upload-service/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33850625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","aws-s3","backend","cloud","cloud-storage","devops","docker","fastapi","file-upload","postgresql","presigned-url","python-backend","rest-api","s3-presigned-url","secure-upload"],"created_at":"2026-06-03T05:31:11.504Z","updated_at":"2026-06-03T05:31:12.529Z","avatar_url":"https://github.com/lyushher.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Secure File Upload \u0026 Presigned URL API\n\n\n![Python](https://img.shields.io/badge/python-3.11-blue)\n![Docker](https://img.shields.io/badge/docker-ready-blue)\n![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)\n![Contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg)\n\n\nThis project provides a scalable backend service for securely uploading files using **presigned URLs** to AWS S3, backed by a PostgreSQL database and built with FastAPI. \n\nIt is designed with clean architecture, production-readiness, and developer ergonomics in mind—ideal for real-world SaaS, internal tools, and cloud-native deployments.\nThe service is containerized using Docker and supports environment-based configuration for seamless deployment.\n\n## 📌 Project Objectives\n\n- Enable **secure and efficient file uploads** to AWS S3 without exposing backend credentials.\n- Provide **presigned URL generation** for direct browser uploads (PUT) and downloads (GET).\n- Store metadata about uploaded files in a **PostgreSQL database**.\n- Ensure clean **separation of concerns** between API layer, service logic, and database models.\n- Allow **environment-based configuration** for deployment across different environments (local, staging, production).\n- Versioned schema migrations powered by **Alembic** for consistent database evolution.\n- Ensure reproducible deployment across environments using **Docker** and **Docker Compose**.\n\n---\n\n## Tech Stack\n\n| Layer           | Technology                          |\n|----------------|--------------------------------------|\n| Framework       | [FastAPI](https://fastapi.tiangolo.com/) |\n| Cloud Storage   | [AWS S3](https://aws.amazon.com/s3/)       |\n| Database        | PostgreSQL + SQLAlchemy             |\n| Migrations      | Alembic                             |\n| Environment     | pydantic-settings + `.env` file     |\n| Containerization| Docker \u0026 docker-compose             |\n| Presigned URLs  | Boto3 (`put_object`, `get_object`)  |\n\n---\n\n## Directory Structure\n\n```\nfile-upload-service/\n│\n├── app/\n│   ├── main.py                  # FastAPI entrypoint\n│   ├── init_db.py               # Script to initialize DB tables\n│   ├── core/\n│   │   ├── database.py          # SQLAlchemy engine, session, Base\n│   │   ├── settings.example.py  # Sample environment config\n│   │   └── validators.py        # Pydantic field-level validators\n│   └── services/\n│       └── s3.py                # S3 utils: presigned URLs, path logic\n│\n├── migrations/                  # Alembic migrations folder\n│\n├── Dockerfile\n├── docker-compose.yml\n├── requirements.txt\n└── README.md \n```\n\n---\n\n## Features\n\n- Generate presigned PUT/GET URLs with content-type control\n- Enforce allowed MIME types via Pydantic\n- Use UUID and date-based pathing for S3 object keys\n- Secure `.env`-based configuration (hidden via `.gitignore`)\n- Database metadata persistence (optional extension)\n- Dockerized environment for dev and prod parity\n- Alembic-powered versioned migrations\n\n---\n\n## ⚙️ Environment Configuration\n\nCreate a `.env` file in the project root (ignored by Git):\n\n```bash\n# .env (Not committed to Git)\nAWS_ACCESS_KEY_ID=your_key\nAWS_SECRET_ACCESS_KEY=your_secret\nAWS_REGION=eu-central-1\nS3_BUCKET=my-upload-bucket\n\nPOSTGRES_USER=postgres\nPOSTGRES_PASSWORD=postgres\nPOSTGRES_DB=file_upload_service\nPOSTGRES_HOST=db\nPOSTGRES_PORT=5432\n\nALLOWED_ORIGINS=http://localhost:3000\nMAX_UPLOAD_MB=25\n```\n\nYou can find a sample at:\n```txt\napp/core/settings.example.py\n```\n\n## Running with Docker\n\n1. Build and Start Containers\n\n```bash\ndocker-compose up --build\n```\n\n2. Access the API Docs\n\nOnce running, go to:\n\n```bash\nhttp://localhost:8000/docs\n```\n\nto access the interactive Swagger UI provided by FastAPI.\n\n---\n\n## API Endpoints\n| Method | Endpoint              | Description                |\n|--------|-----------------------|----------------------------|\n| GET    | /health               | Basic health check         |\n| POST   | /presign/upload       | Generate presigned PUT URL |\n| GET    | /presign/download/{key} | Generate presigned GET URL  |\n\nEach endpoint validates file types and request payload using [Pydantic](https://docs.pydantic.dev/latest/) schemas.\n\nExample Request (Presigned Upload)\n```http\nPOST /presign/upload\nContent-Type: application/json\n```\n```json\n{\n  \"filename\": \"document.pdf\",\n  \"content_type\": \"application/pdf\"\n}\n```\nExample Response:\n```json\n{\n  \"url\": \"https://your-bucket.s3.amazonaws.com/2025/08/14/uuid-filename.pdf\",\n  \"key\": \"2025/08/14/uuid-filename.pdf\"\n}\n```\nSecurity Note: \n\u003e **Ensure that only validated content types are accepted** to avoid uploading malicious files.\n\n---\n\n## Alembic Migrations\n\nInitialize and apply migrations:\n\n```bash\nalembic init migrations\nalembic revision --autogenerate -m \"Initial schema\"\nalembic upgrade head\n```\n\nMigrations are tracked in the `migrations/versions/` folder.\n\n\n## Security Notes\n\n- No AWS credentials are ever exposed to the client.\n- Only allowed MIME types can be uploaded.\n- File names are sanitized to remove dangerous characters.\n- Large files are capped via environment-configured byte limit.\n- Only presigned requests can access/upload files.\n\n\n## Checklist for Production\n\n- Do not commit .env — only settings.example.py\n- Set DEBUG=False in production\n- Secure AWS permissions via least-privilege IAM role\n- Use HTTPS-only endpoints in production\n\n## ⚠️ Disclaimer\n\n\u003e **Note:** This project is a public showcase of the overall architecture, structure, and functionality of a secure file upload backend.\nThe actual production implementation, including critical components, secrets, and internal logic, is private and maintained in a separate repository.\n\n- All credentials and sensitive configurations are omitted or replaced with mock values.\n- Any resemblance to production behavior is for demonstration purposes only.\n- This repository is not intended for direct deployment in production environments without further security validation.\n\n## License\n\nThis project is licensed under the **Apache License 2.0**.  \nYou may use, modify, and distribute it in accordance with the terms of the license.\n\nSee the full [LICENSE](./LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyushher%2Ffile-upload-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flyushher%2Ffile-upload-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyushher%2Ffile-upload-service/lists"}