{"id":25367974,"url":"https://github.com/mizanmahi/reelserver","last_synced_at":"2026-04-06T02:34:16.186Z","repository":{"id":275569465,"uuid":"925145428","full_name":"mizanmahi/reelserver","owner":"mizanmahi","description":"ReelShare Server is a high-performance, TypeScript-based backend designed to power a seamless video-sharing platform.","archived":false,"fork":false,"pushed_at":"2025-02-19T14:45:45.000Z","size":226,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-09T05:44:18.535Z","etag":null,"topics":["docker","express","nginx","nodejs","postgresql","promethus","redis","s3-bucket","typescript","winston"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mizanmahi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-31T10:14:56.000Z","updated_at":"2025-02-19T14:45:48.000Z","dependencies_parsed_at":"2025-02-18T17:41:36.593Z","dependency_job_id":null,"html_url":"https://github.com/mizanmahi/reelserver","commit_stats":null,"previous_names":["mizanmahi/reelserver"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mizanmahi/reelserver","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mizanmahi%2Freelserver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mizanmahi%2Freelserver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mizanmahi%2Freelserver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mizanmahi%2Freelserver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mizanmahi","download_url":"https://codeload.github.com/mizanmahi/reelserver/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mizanmahi%2Freelserver/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31457722,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T21:22:52.476Z","status":"online","status_checked_at":"2026-04-06T02:00:07.287Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","express","nginx","nodejs","postgresql","promethus","redis","s3-bucket","typescript","winston"],"created_at":"2025-02-15T00:23:51.513Z","updated_at":"2026-04-06T02:34:16.170Z","avatar_url":"https://github.com/mizanmahi.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# **ReelShare Server**\n\n---\n\n## **Table of Contents**\n\n1. [Project Description and Features](#project-description-and-features)\n2. [Setup Instructions](#setup-instructions)\n3. [API Documentation](#api-documentation)\n4. [Database Design](#database-design)\n5. [Architecture Diagram](#architecture-diagram)\n6. [Technical Decision Explanation](#technical-decision-explanation)\n7. [Performance Optimization Strategy](#performance-optimization-strategy)\n8. [Deployment Strategy](#deployment-strategy)\n9. [System Design Decisions](#system-design-decisions)\n10.   [Possible Improvements](#possible-improvements)\n11.   [Conclusion](#conclusion)\n\n---\n\n## **Project Description and Features**\n\nReelShare Server is a high-performance, TypeScript-based backend designed to power a seamless video-sharing platform. It enables users to upload, process, and stream short-form videos efficiently while ensuring security, scalability, and observability.\n\nBuilt with modern technologies like Node.js, PostgreSQL, Redis, ~~MinIO~~, and Docker, it provides essential video processing features, including on-the-fly compression, thumbnail extraction, caching, and rate limiting. The backend is optimized for performance with background workers, log rotation, and API monitoring using Prometheus and Grafana.\n\nWith a Dockerized architecture and CI/CD deployment on AWS EC2, ReelShare Server ensures reliability and easy scalability. The platform is ideal for video-centric applications, allowing developers to extend and integrate new features effortlessly. 🚀\n\n### **Key Features**\n\n-  **TypeScript-Based Application**: Built with TypeScript for type safety, scalability, and maintainability.\n-  **User Authentication**: Basic user authentication using JSON Web Tokens (JWT) for secure access.\n-  **Video Uploading**: Seamless video upload functionality.\n-  **Video Compression**: On-the-fly video compression to optimize storage and bandwidth usage.\n-  **Thumbnail Extraction**: Automatic extraction of thumbnails from uploaded videos for preview purposes.\n-  **Log Generation with Rotation**: Comprehensive logging with log rotation to manage disk space efficiently.\n-  **Caching with Redis**: High-performance caching using Redis to reduce database load and improve response times.\n-  **Rate Limiting**: Built-in rate limiting to prevent abuse and protect against DDoS attacks.\n-  **API Monitoring**: Integrated monitoring using Prometheus and Grafana for real-time performance insights.\n-  **Nginx Reverse Proxy**: Configured Nginx as a reverse proxy to efficiently route incoming requests to the appropriate application containers, ensuring high availability and scalability.\n\n---\n\n## **Setup Instructions**\n\nTo run ReelServer, you need to have Docker installed on your system. Follow the steps below to set up and run the application.\n\n### **Prerequisites**\n\n-  Docker installed on your system.\n-  Docker Compose installed.\n\n### **Steps to Run the Application**\n\n1. **Clone the Repository**:\n\n   ```\n   git clone https://github.com/mizanmahi/reelserver.git\n   cd reelserver\n   ```\n\n2. **Set Up Environment Variables**:\n   Create a `.env` file in the root directory and add the necessary environment variables. Refer to the `docker-compose.yml` file for required variables. The following environment variables are required for AWS S3 integration:\n\n   ```\n   AWS_ACCESS_KEY=your_access_key\n   AWS_SECRET_ACCESS_KEY=your_secret_access_key\n   AWS_S3_REGION=your_aws_region (e.g., ap-southeast-1)\n   AWS_S3_BUCKET_NAME=your_bucket_name\n   ```\n\n3. **Run Docker Compose**:\n   Start all the services using Docker Compose:\n\n   ```\n   docker-compose up -d\n   ```\n\n4. **Verify Running Containers**:\n   Check the status of the running containers:\n\n   ```\n   docker-compose ps\n   ```\n\n### **Ports and Services**\n\n-  **PostgreSQL**: `5431` (Host) → `5432` (Container)\n-  ~~**MinIO**: `9000` (API) and `9001` (Console)~~\n-  **Redis**: `6379` (Redis) and `8001` (Redis Stack for Insight)\n-  **Prometheus**: `9090`\n-  **Application**: `5000`\n\n---\n\n## **API Documentation**\n\nExplore the API endpoints and test them using the provided Postman documentation:  \n[Postman Documentation Link](https://documenter.getpostman.com/view/28371413/2sAYX3s4Sk)\n\n---\n\n## **Database Design**\n\nThe database design consists of three main models: `User`, `Video`, and `Engagement`. Below is an explanation of the relationships between these models, followed by a diagram.\n\n### Relationships\n\n1. **User**:\n\n   -  A `User` can create multiple `Videos` (one-to-many relationship).\n   -  A `User` can engage with multiple `Videos` through the `Engagement` model (many-to-many relationship).\n\n2. **Video**:\n\n   -  A `Video` is uploaded by a single `User` (many-to-one relationship).\n   -  A `Video` can have multiple `Engagements` (one-to-many relationship).\n   -  A `Video` contains metadata (stored as JSON) and engagement metrics like `viewCount` and `likeCount`.\n\n3. **Engagement**:\n\n   -  The `Engagement` model represents interactions between a `User` and a `Video` (e.g., likes, views, etc.).\n   -  It is a join table that connects `User` and `Video` in a many-to-many relationship.\n   -  Each engagement record is uniquely identified by a composite key (`videoId` and `userId`).\n\n### Diagram\n\nBelow is a diagram illustrating the relationships between the `User`, `Video`, and `Engagement` models:\n\n```mermaid\nerDiagram\n    User ||--o{ Video : \"uploads\"\n    User ||--o{ Engagement : \"engages\"\n    Video ||--o{ Engagement : \"has\"\n    User {\n        String id\n        String name\n        String email\n        String contact\n        String password\n        DateTime createdAt\n        DateTime updatedAt\n    }\n    Video {\n        String id\n        String videoUrl\n        String title\n        String description\n        String thumbnail\n        String uploaderId\n        Int viewCount\n        Int likeCount\n        DateTime createdAt\n        DateTime updatedAt\n        Json metadata\n    }\n    Engagement {\n        String videoId\n        String userId\n        DateTime createdAt\n    }\n```\n\n---\n\n## **Architecture Diagram**\n\nBelow is a high-level architecture diagram of ReelServer:\n\n```mermaid\ngraph TD\n    A[Client Request] --\u003e B[Node.js Server]\n    B --\u003e C{Cache Check}\n    C -- Cache Hit --\u003e D[Return Cached Data to Client]\n    C -- Cache Miss --\u003e E[Query Postgres Database]\n    E --\u003e F{Is Video Processing Required?}\n    F -- Yes --\u003e G[Offload to Node.js Worker Thread]\n    G --\u003e H[Process Video]\n    H --\u003e I[Update Cache]\n    F -- No --\u003e I\n    I --\u003e J[Return Data to Client]\n```\n\n---\n\n## **Technical Decision Explanation**\n\n1. **TypeScript**: Chosen for its static typing, which reduces runtime errors and improves code maintainability.\n2. **JWT for Authentication**: Provides a stateless and scalable way to handle user authentication.\n3. **PostgreSQL**: Selected for its reliability, robustness, and support for complex queries. It is well-suited for handling structured data like user information, video metadata, engagements and logs.\n4. **FFmpeg for Video Processing**: Used for video compression and thumbnail extraction due to its versatility, performance, and wide support for various video formats and codecs.\n5. **Redis for Caching**: Enhances performance by reducing database load for frequently accessed data.\n6. **~~MinIO for Storage~~**: ~~Offers an S3-compatible object storage solution, ideal for handling large video files.~~ **AWS S3 for Storage**: Provides scalable and reliable object storage for handling large video files.\n7. **Prometheus and Grafana**: Provides robust monitoring and visualization capabilities for API performance and health.\n8. **Docker and Docker Compose**: Simplifies deployment and ensures consistency across development, testing, and production environments.\n\n---\n\n## **Performance Optimization Strategy**\n\n1. **Caching**: Use Redis to cache frequently accessed data, reducing database queries.\n2. **Rate Limiting**: Implement rate limiting to prevent abuse and ensure fair usage of resources.\n3. **Video Compression**: Compress videos during upload to save storage and reduce bandwidth usage.\n4. **Log Rotation**: Manage log files efficiently to prevent disk space issues.\n5. **Asynchronous Processing**: Offload tasks like video compression and thumbnail extraction to background workers. Used multi-threading with worker_threads. 🔄\n6. **Fine-Tuning Compression**: Reduce payload size using gzip or Brotli compression. ⚡\n\n---\n\n## **Deployment Strategy**\n\nReelServer is deployed on an EC2 instance using GitHub Actions for CI/CD. Below is the deployment workflow:\n\n### **Deployment Workflow**\n\n1. **Trigger**: The workflow is triggered on a push or pull request to the `main` branch.\n2. **SSH Configuration**: The workflow configures SSH access to the EC2 instance.\n3. **File Transfer**: The application files are transferred to the EC2 instance using `rsync`.\n4. **Docker Setup**: Ensures Docker and Docker Compose are installed on the EC2 instance.\n5. **Container Management**: Stops existing containers, prunes unused resources, and starts new containers using Docker Compose.\n6. **Nginx Reverse Proxy**: Configures Nginx as a reverse proxy to route incoming traffic on port 80 to the application running in the Docker container.\n\n### **Accessing the Deployed Server**\n\nThe application is deployed at:... **Live URL:** `https://devsskills.com`  \nNote: The instance reachability check may fail at any time due to limited resources on free-tier instances.\n\n#### **Video Hosting Disclaimer**\n\nThe videos are now hosted on AWS S3.\n~~The videos are hosted on MinIO locally on the server. This means that they may not play when accessed via `localhost`. To properly access videos, use the public IP of the EC2 instance instead of `localhost`.~~\n\n~~Example:~~\n\n-  ~~❌ `http://localhost:9000/videos/videos/compressed_1738561984341_IMG_2468.mp4`~~\n-  ~~✅ `http://serverPublicIp:9000/videos/videos/compressed_1738561984341_IMG_2468.mp4`~~\n\nFor a better experience, clone and run both the frontend and backend on your machine. Below is the frontend repo link:\n\n**Frontend Repository:** [Frontend Repo Link](https://github.com/mizanmahi/reelclient.git)\n\n### **Deployment YAML**\n\nFor the complete `deploy.yml` file, refer to the [deploy.yml file](https://github.com/mizanmahi/reelserver/blob/main/.github/workflows/deploy.yml) in the repository.\n\n### **Possible Improvements**\n\nThe current deployment can be improved in several ways:\n\n1. **HTTP Live Streaming (HLS)**: Implementing HLS can allow efficient streaming of video data instead of direct file serving.\n2. **NGINX with SSL Certificate**: Deploying the application behind an NGINX reverse proxy with an SSL certificate (Let's Encrypt) would improve security and allow HTTPS access.\n3. **Auto-Scaling with Load Balancer**: Using AWS Auto Scaling with an Elastic Load Balancer (ELB) can handle traffic spikes effectively.\n4. **~~S3 \u0026 CloudFront for Video Hosting~~**: ~~Instead of hosting videos on MinIO locally, using AWS S3 with CloudFront CDN would improve video delivery speed and availability.~~ **CloudFront for Video Delivery**: Using AWS S3 with CloudFront CDN would improve video delivery speed and availability.\n\nThese improvements can enhance performance, scalability, and security of the deployment. Let me know if you want to implement any of them! 🚀\n\n## **System Design Decisions**\n\nFor the complete System Design Decisions refer to the [SystemDesignDecisions.md file](https://github.com/mizanmahi/reelserver/blob/main/SystemDesignDecisions.md).\n\n---\n\n## **Conclusion**\n\nReelServer is a powerful and scalable solution for managing video workflows. With its robust features, efficient architecture, and seamless deployment process, it is designed to meet the needs of modern video processing applications.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmizanmahi%2Freelserver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmizanmahi%2Freelserver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmizanmahi%2Freelserver/lists"}