{"id":17780697,"url":"https://github.com/swarajkumarsingh/turbo-deploy","last_synced_at":"2025-08-17T08:04:10.278Z","repository":{"id":258948755,"uuid":"867034185","full_name":"swarajkumarsingh/turbo-deploy","owner":"swarajkumarsingh","description":"Turbo Deploy - Deploy your application in seconds 🚀","archived":false,"fork":false,"pushed_at":"2025-04-28T06:49:23.000Z","size":843,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-28T07:40:08.328Z","etag":null,"topics":["aws-elasticache","aws-rds","deployment","go","grafana","pgadmin","postgresql","prometheus","redis"],"latest_commit_sha":null,"homepage":"https://github.com/swarajkumarsingh/turbo-deploy","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swarajkumarsingh.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-03T10:33:39.000Z","updated_at":"2025-04-28T06:49:27.000Z","dependencies_parsed_at":"2024-11-13T16:28:23.297Z","dependency_job_id":"7a866e6b-5f05-4168-9a7d-390cfa5094d7","html_url":"https://github.com/swarajkumarsingh/turbo-deploy","commit_stats":{"total_commits":10,"total_committers":1,"mean_commits":10.0,"dds":0.0,"last_synced_commit":"1bdb2208d6c5c6beca81dfcdf69eb178571e8473"},"previous_names":["swarajkumarsingh/turbo-deploy"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/swarajkumarsingh/turbo-deploy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swarajkumarsingh%2Fturbo-deploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swarajkumarsingh%2Fturbo-deploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swarajkumarsingh%2Fturbo-deploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swarajkumarsingh%2Fturbo-deploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swarajkumarsingh","download_url":"https://codeload.github.com/swarajkumarsingh/turbo-deploy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swarajkumarsingh%2Fturbo-deploy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270820793,"owners_count":24651534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-elasticache","aws-rds","deployment","go","grafana","pgadmin","postgresql","prometheus","redis"],"created_at":"2024-10-27T03:03:34.399Z","updated_at":"2025-08-17T08:04:10.230Z","avatar_url":"https://github.com/swarajkumarsingh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Turbo Deploy 🚀\n\nTurbo Deploy is an enterprise-grade deployment platform that automates the entire process of deploying React/Vite applications. With a single Git URL, it orchestrates building, hosting, and serving your application through a sophisticated AWS infrastructure pipeline. This project is built using Go, Node.js, PostgreSQL, and several AWS services, including RDS, S3, ElasticCache, SES, SQS, ECS, and EC2.\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Docker](https://img.shields.io/badge/Docker-Enabled-blue)](https://www.docker.com/)\n[![AWS](https://img.shields.io/badge/AWS-Powered-orange)](https://aws.amazon.com/)\n\n## Table of Contents 📑\n\n- [Features](#features)\n- [Architecture](#architecture)\n- [Prerequisites](#prerequisites)\n- [Project Structure](#Project-Structure)\n- [Installation](#installation)\n- [Configuration](#configuration)\n- [Usage](#usage)\n- [API Reference](#api-reference)\n- [Infrastructure](#infrastructure)\n- [Monitoring](#monitoring)\n- [Security](#security)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Features\n\n### Core Capabilities\n- **Automated Deployments**\n  - Zero-configuration deployment pipeline\n  - Automatic build optimization\n  - Smart caching strategies\n  - Rolling updates with zero downtime\n\n### Build System\n- **Isolated Build Environment**\n  - Containerized builds using AWS ECS\n  - Concurrent build support\n  - Custom build parameter configuration\n  - Build artifact caching\n\n### Hosting \u0026 Delivery\n- **S3-Powered Static Hosting**\n  - Automatic asset optimization\n  - Content compression\n  - Cache control headers\n  - Custom domain support\n\n### Monitoring \u0026 Logging\n- **Comprehensive Observability**\n  - Real-time deployment logs\n  - Performance metrics\n  - Error tracking\n  - Resource utilization monitoring\n\n### Security\n- **Enterprise-Grade Security**\n  - SSL/TLS encryption\n  - Sanitization of Build Commands\n\n## Architecture\n\n### System Components\n\n#### Frontend Layer (yet to be built)\n- React/Vite application hosting\n- Custom domain management\n- SSL/TLS termination\n- Static asset serving\n\n#### Application Services\n\n1. **Build Server (Nodejs)**\n   - Handles build requests\n   - Manages ECS task lifecycle\n   - Coordinates with S3 for artifact storage\n   - Implements build caching\n\n2. **Reverse Proxy (Go, Node.js)**\n   - Routes requests to appropriate S3 buckets\n   - Handles custom domain mapping\n   - Manages SSL/TLS certificates\n   - Implements caching strategies\n\n3. **Log Consumer Service (Go)**\n   - Processes SQS messages\n   - Aggregates build logs\n   - Handles log retention\n   - Provides real-time log streaming\n\n4. **Email Service (Go)**\n   - Manages notification templates\n   - Handles deployment notifications\n   - Processes email queues\n   - Tracks delivery status\n\n#### Infrastructure Services\n\n1. **Database Layer (PostgreSQL)**\n   - Deployment records\n   - User management\n   - Configuration storage\n   - Audit logs\n\n2. **Cache Layer (Redis)**\n   - Session management\n   - Rate limiting\n   - Build cache\n   - Temporary data storage\n\n3. **Message Queue (SQS)**\n   - Service communication\n   - Event processing\n   - Log aggregation\n   - Email notifications\n\n4. **Storage Layer (S3)**\n   - Static file hosting\n   - Build artifacts\n   - Log archives\n   - Backup storage\n\n## Prerequisites\n\n### System Requirements\n- Docker Engine 20.10+\n- Docker Compose 2.0+\n- Node.js 16.x+\n- Go 1.19+\n- PostgreSQL 13+\n- Redis 6+\n\n### AWS Requirements\n- AWS Account with admin access\n- Configured AWS CLI\n- Required AWS services enabled:\n  - EC2\n  - ECS\n  - S3\n  - RDS\n  - ElastiCache\n  - SQS\n  - SES\n  - IAM\n\n### Network Requirements\n- Public IP address\n- Domain name\n- SSL certificate\n\n\n## Project Structure\n\nThe Turbo-Deploy project is built with a modular structure to ensure scalability, maintainability, and ease of development. Below is a breakdown of the key directories and their roles:\n\n### **/controller**  \nContains the logic for handling incoming requests and orchestrating business processes.  \n- **prometheus**: Custom Prometheus metrics for real-time monitoring and observability.  \n- **user**: Handles user-specific operations like authentication, user profile management, and access control.  \n\n### **/docs**  \nHouses project documentation, API references, and technical guides.  \n\n### **/errorHandler**  \nCentralized error-handling mechanism used throughout the application to ensure consistency and structured error responses.  \n\n### **/functions**  \nA collection of utility modules to handle common tasks across services:  \n- **general**: Generic utilities used in various modules.  \n- **logger**: Custom logging utilities for structured and level-based logging.  \n- **retry**: Helper functions for implementing retry logic for failed operations.  \n- **tracer**: OpenTelemetry tracing utilities for distributed system observability.  \n- **uploader**: Tools for handling file uploads to S3.  \n- **validator**: Input validation functions to enforce data integrity.  \n\n### **/infra**  \nInfrastructure-specific modules that abstract interactions with external services:  \n- **db**: PostgreSQL database connection and query abstraction.  \n- **redis**: Integration with AWS ElasticCache for caching.  \n- **s3**: S3 bucket management and operations for file storage.  \n- **ses**: Email sending functionalities via AWS SES.  \n- **sqs**: Integration with AWS SQS for event queue management.  \n\n### **/migrations**  \nScripts for managing database schema migrations. Ensures smooth transitions when modifying the database schema.  \n\n### **/models**  \nDefines the data models and business logic for various entities:  \n- **deployment**: Manages deployment processes and state.  \n- **deployment_log**: Tracks deployment history and logs.  \n- **error**: Standardized error model for the system.  \n- **project**: Handles project-related data and logic.  \n- **user**: User-specific data and operations.  \n\n### **/routes**  \nDefines API routes and their mapping to controller functions. Acts as the entry point for incoming requests.  \n\n### **/services**  \nContains microservices, each performing specific roles:  \n- **build-server**: Executes ECS tasks to clone repositories, build projects, and upload artifacts to S3.  \n- **email-consumer**: Listens to SQS queues and sends email notifications using AWS SES.  \n- **logs-sqs-consumer**: Processes logs/events from the SQS queues and updates deployment logs or metrics.  \n- **reverse-proxy**: Dynamically serves files from S3 based on subdomains.  \n\n---\n\n\n## Installation\n\n### Local Development Setup\n\n1. **Clone the Repository**\n```bash\ngit clone https://github.com/swarajkumarsingh/turbo-deploy.git\ncd turbo-deploy\n```\n\n3. **Env Setup**\n```bash\n# edit the run.local.sh files accordingly to your config\nnano ./run.local.sh\n```\n\n2. **Run .sh file**\n```bash\n./run.local.sh\n```\n\n3. **Run all services(consumers)**\n```bash\n# migrate to root/services\nmake start\n```\n\n## Configuration\n\n### Environment Variables\n\n```env\nDB_HOST=http://127.0.0.1/\nDB_PORT=5432\nDB_USER=user\nDB_PASSWORD=postgres\nDB_NAME=turbo-deploy\nREDIS_HOST=\nREDIS_PORT=\nREDIS_USER=\nREDIS_PASSWORD=\nSENTRY_DSN=\nDD_AGENT_HOST=\nS3_BUCKET=\n\nSTAGE=dev\nAWS_SQS_URL=\nAWS_REGION=\nAWS_ACCESS_KEY=\nAWS_SECRET_ACCESS_KEY=\nDB_URL=\n```\n\n\n## Usage\n\n### API Examples\n- check .postman folder postman.json --\u003e import into Postman Client\n\n## Infrastructure 🏗️\n\n### AWS Resources\n\n1. **Compute**\n   - ECS Cluster\n   - EB elastic beanstalk\n   - EC2 Auto Scaling Group\n   - Application Load Balancer\n   - ECR - Registry for code storage\n\n2. **Storage**\n   - S3 Buckets\n   - RDS Instance\n   - ElastiCache Cluster\n\n3. **Networking**\n   - VPC\n   - Subnets\n   - Security Groups\n\n4. **Services**\n   - SES Email Service\n   - SQS Queue Service\n\n### AWS TaskDefinition JSON\n\n```hcl\n{\n  \"taskDefinitionArn\": \"arn:aws:ecs:ap-north-99:xxxx:task-definition/builder-task:1\",\n  \"containerDefinitions\": [\n    {\n      \"name\": \"my-image\",\n      \"image\": \"xxxx.dkr.ecr.ap-south-1.amazonaws.com/build-server:latest\",\n      \"cpu\": 0,\n      \"portMappings\": [],\n      \"essential\": true,\n      \"environment\": [],\n      \"environmentFiles\": [],\n      \"mountPoints\": [],\n      \"volumesFrom\": [],\n      \"ulimits\": [],\n      \"logConfiguration\": {\n        \"logDriver\": \"awslogs\",\n        \"options\": {\n          \"awslogs-group\": \"/ecs/thor-flask\",\n          \"mode\": \"non-blocking\",\n          \"awslogs-create-group\": \"true\",\n          \"max-buffer-size\": \"25m\",\n          \"awslogs-region\": \"ap-south-1\",\n          \"awslogs-stream-prefix\": \"ecs\"\n        },\n        \"secretOptions\": []\n      },\n      \"systemControls\": []\n    }\n  ],\n  \"family\": \"thor-flask\",\n  \"executionRoleArn\": \"arn:aws:iam::xxxx:role/ecsTaskExecutionRole\",\n  \"networkMode\": \"awsvpc\",\n  \"revision\": 1,\n  \"volumes\": [],\n  \"status\": \"ACTIVE\",\n  \"requiresAttributes\": [\n    {\n      \"name\": \"com.amazonaws.ecs.capability.logging-driver.awslogs\"\n    },\n    {\n      \"name\": \"ecs.capability.execution-role-awslogs\"\n    },\n    {\n      \"name\": \"com.amazonaws.ecs.capability.ecr-auth\"\n    },\n    {\n      \"name\": \"com.amazonaws.ecs.capability.docker-remote-api.1.19\"\n    },\n    {\n      \"name\": \"com.amazonaws.ecs.capability.docker-remote-api.1.28\"\n    },\n    {\n      \"name\": \"ecs.capability.execution-role-ecr-pull\"\n    },\n    {\n      \"name\": \"com.amazonaws.ecs.capability.docker-remote-api.1.18\"\n    },\n    {\n      \"name\": \"ecs.capability.task-eni\"\n    },\n    {\n      \"name\": \"com.amazonaws.ecs.capability.docker-remote-api.1.29\"\n    }\n  ],\n  \"placementConstraints\": [],\n  \"compatibilities\": [\"EC2\", \"FARGATE\"],\n  \"requiresCompatibilities\": [\"FARGATE\"],\n  \"cpu\": \"1024\",\n  \"memory\": \"3072\",\n  \"runtimePlatform\": {\n    \"cpuArchitecture\": \"XxxXX\",\n    \"operatingSystemFamily\": \"LINUX\"\n  },\n  \"registeredAt\": \"2024-12-xxxx:36.078Z\",\n  \"registeredBy\": \"arn:aws:iam::xxxx:root\",\n  \"enableFaultInjection\": false,\n  \"tags\": []\n}\n\n```\n\n### Resource Specifications\n\n```hcl\n# ECS Task Definition\nresource \"aws_ecs_task_definition\" \"build\" {\n  family                   = \"build\"\n  requires_compatibilities = [\"FARGATE\"]\n  network_mode            = \"awsvpc\"\n  cpu                     = 1024\n  memory                  = 2048\n  \n  container_definitions = jsonencode([\n    {\n      name  = \"build\"\n      image = \"${aws_ecr_repository.build.repository_url}:latest\"\n      # ... additional configuration\n    }\n  ])\n}\n```\n\n## Monitoring\n\n### Available Metrics\n\n1. **System Metrics**\n   - CPU Usage\n   - Memory Usage\n   - Network I/O\n   - Disk Usage\n\n2. **Application Metrics**\n   - Build Success Rate\n   - Build Duration\n   - Cache Hit Rate\n   - Error Rate\n\n3. **Business Metrics**(yet to be built)\n   - Active Deployments\n   - Total Users\n   - Resource Consumption\n   - Cost Analysis\n\n### Prometheus Configuration\n\n```yaml\nglobal:\n  scrape_interval: 15s\n\nscrape_configs:\n  - job_name: 'turbo-deploy'\n    static_configs:\n      - targets: ['localhost:9090']\n```\n\n## Security\n\n### Authentication\n\n- JWT-based authentication\n- Rate limiting\n- IP whitelisting\n\n### Command Sanitization\n\n- npm build sanitization for security\n\n## Contributing\n\n### Development Process\n\n1. Fork the repository\n2. Create a feature branch\n3. Implement changes\n4. Add tests\n5. Submit pull request\n\n### Code Standards\n\n- Follow Go style guide\n- Use ESLint for JavaScript\n- Write unit tests\n- Document changes\n\n## License\n\nMIT License\n\nCopyright (c) 2024 Swaraj Kumar Singh\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n\n## Support\n\n- Email: sswaraj169@gmail.com\n- Medium: [Join our community](https://swarajkumarsingh.medium.com/)\n- Documentation: [Read the docs](https://github.com/swarajkumarsingh/turbo-deploy/blob/main/README.md)\n- Issues: [GitHub Issues](https://github.com/username/turbo-deploy/issues)\n\n---\n\nMade with ❤️ by Swaraj Kumar Singh","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswarajkumarsingh%2Fturbo-deploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswarajkumarsingh%2Fturbo-deploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswarajkumarsingh%2Fturbo-deploy/lists"}