{"id":31078179,"url":"https://github.com/setavitiki/shopflow-platform","last_synced_at":"2026-04-12T17:35:42.276Z","repository":{"id":314545233,"uuid":"1051571383","full_name":"setavitiki/shopflow-platform","owner":"setavitiki","description":"Cloud-native e-commerce platform with 5 microservices deployed on AWS EKS using Istio service mesh, automated CI/CD with GitHub Actions, and comprehensive observability.","archived":false,"fork":false,"pushed_at":"2025-09-13T04:31:29.000Z","size":1036,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-13T06:26:57.735Z","etag":null,"topics":["aws","aws-eks","ci-cd","cloud-native","containerization","devops","distributed-systems","docker","github-actions","istio","kubernet","microservices","nodejs","service-mesh"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/setavitiki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-06T09:19:31.000Z","updated_at":"2025-09-13T04:29:32.000Z","dependencies_parsed_at":"2025-09-13T06:37:29.227Z","dependency_job_id":null,"html_url":"https://github.com/setavitiki/shopflow-platform","commit_stats":null,"previous_names":["setavitiki/shopflow-platform"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/setavitiki/shopflow-platform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/setavitiki%2Fshopflow-platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/setavitiki%2Fshopflow-platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/setavitiki%2Fshopflow-platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/setavitiki%2Fshopflow-platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/setavitiki","download_url":"https://codeload.github.com/setavitiki/shopflow-platform/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/setavitiki%2Fshopflow-platform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275384116,"owners_count":25454910,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-eks","ci-cd","cloud-native","containerization","devops","distributed-systems","docker","github-actions","istio","kubernet","microservices","nodejs","service-mesh"],"created_at":"2025-09-16T08:06:00.052Z","updated_at":"2026-04-12T17:35:42.261Z","avatar_url":"https://github.com/setavitiki.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud-Native ShopFlow Microservices Platform Platform\n\n[![AWS EKS](https://img.shields.io/badge/AWS-EKS-orange.svg)](https://aws.amazon.com/eks/)\n[![Istio](https://img.shields.io/badge/Service%20Mesh-Istio-blue.svg)](https://istio.io/)\n[![Kubernetes](https://img.shields.io/badge/Orchestration-Kubernetes-326CE5.svg)](https://kubernetes.io/)\n[![GitHub Actions](https://img.shields.io/badge/CI%2FCD-GitHub%20Actions-2088FF.svg)](https://github.com/features/actions)\n\nA production-ready, cloud-native e-commerce platform built with microservices architecture, deployed on AWS EKS with Istio service mesh. This project demonstrates advanced DevOps practices, container orchestration, and modern cloud-native engineering patterns.\n\n## Architecture Overview\n\nThis platform implements a sophisticated microservices ecosystem designed for scalability, resilience, and maintainability:\n\n```\n┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐\n│  GitHub Actions │────│  AWS EKS Cluster │────│   Istio Mesh    │\n│  CI/CD Pipeline │    │  + Auto Scaling  │    │   Traffic Mgmt  │\n└─────────────────┘    └──────────────────┘    └─────────────────┘\n         │                        │                        │\n         ▼                        ▼                        ▼\n┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐\n│  ECR Registry   │    │  5 Microservices │    │  Load Balancer  │\n│  Container Imgs │    │  + Envoy Proxies │    │  External Acc.  │\n└─────────────────┘    └──────────────────┘    └─────────────────┘\n```\n\n### Core Components\n- **5 Microservices**: Authentication, Product Catalog, Order Management, Payment Processing, Notifications\n- **Container Orchestration**: AWS Elastic Kubernetes Service (EKS) with managed node groups\n- **Service Mesh**: Istio 1.27 with Envoy proxy sidecars for traffic management\n- **CI/CD Pipeline**: GitHub Actions with automated testing, building, and deployment\n- **Container Registry**: AWS Elastic Container Registry (ECR) with vulnerability scanning\n- **Networking**: AWS Application Load Balancer with Istio Gateway integration\n\n## Project Objectives \u0026 Learning Outcomes\n\n### Technical Mastery Demonstrated\n- **Cloud-Native Architecture**: Microservices design patterns and distributed system principles\n- **Container Orchestration**: Advanced Kubernetes concepts including resource management and scaling\n- **Service Mesh Integration**: Traffic routing, security policies, and observability\n- **DevOps Engineering**: Automated CI/CD pipelines with proper secrets management\n- **Infrastructure as Code**: Declarative configuration management and GitOps practices\n\n### Production-Ready Features\n- **Zero-Downtime Deployments**: Rolling updates with health checks and readiness probes\n- **Horizontal Scaling**: Auto-scaling capabilities for handling variable load\n- **Security Best Practices**: Container image scanning, secrets management, and network policies\n- **Observability**: Service topology visualization and traffic monitoring\n- **Fault Tolerance**: Circuit breaker patterns and retry mechanisms ready for implementation\n\n## Technology Stack\n\n### Infrastructure \u0026 Platform\n- **Cloud Provider**: Amazon Web Services (AWS)\n- **Kubernetes Distribution**: AWS EKS (Elastic Kubernetes Service)\n- **Service Mesh**: Istio 1.27 with Envoy proxy\n- **Container Runtime**: Docker with multi-stage optimized builds\n- **Load Balancing**: AWS Application Load Balancer + Istio Gateway\n\n### Development \u0026 Operations\n- **Programming Language**: Node.js with Express.js framework\n- **CI/CD Platform**: GitHub Actions with matrix builds\n- **Container Registry**: AWS ECR with automated vulnerability scanning\n- **Version Control**: Git with GitFlow branching strategy\n- **Configuration Management**: Kubernetes manifests with environment-specific overlays\n\n### Observability \u0026 Monitoring\n- **Service Topology**: Kiali dashboard for mesh visualization\n- **Traffic Management**: Istio VirtualServices and DestinationRules\n- **Health Monitoring**: Kubernetes liveness and readiness probes\n- **Logging**: Container-native logging with kubectl integration\n\n## Prerequisites \u0026 Requirements\n\n### Local Development Environment\n```bash\n# Required tools and versions\naws-cli \u003e= 2.0\nkubectl \u003e= 1.28\neksctl \u003e= 0.100\nistioctl \u003e= 1.27\ndocker \u003e= 20.0\ngit \u003e= 2.30\n```\n\n### AWS Account Setup\n- AWS Account with programmatic access\n- IAM user with EKS, ECR, and VPC permissions\n- AWS CLI configured with appropriate credentials\n- Sufficient service limits for EKS cluster creation\n\n### GitHub Repository Configuration\n- Repository with Actions enabled\n- Required secrets configured (detailed in setup section)\n- Branch protection rules (recommended for production)\n\n## Complete Setup Guide\n\n### Phase 1: Infrastructure Provisioning\n\n#### 1.1 EKS Cluster Creation\n```bash\n# Create production-ready EKS cluster\neksctl create cluster \\\n  --name shopflow-cluster \\\n  --region ap-south-1 \\\n  --nodes 2 \\\n  --node-type t3.medium \\\n  --nodes-min 1 \\\n  --nodes-max 4 \\\n  --managed \\\n  --enable-ssm\n\n# Verify cluster access\nkubectl get nodes\nkubectl cluster-info\n```\n\n#### 1.2 Container Registry Setup\n```bash\n# Create ECR repositories for each microservice\nservices=(\"auth-service\" \"product-service\" \"order-service\" \"payment-service\" \"notification-service\")\n\nfor service in \"${services[@]}\"; do\n  aws ecr create-repository \\\n    --repository-name shopflow/$service \\\n    --region ap-south-1 \\\n    --image-scanning-configuration scanOnPush=true\n  echo \"Created repository for $service\"\ndone\n\n# Verify repository creation\naws ecr describe-repositories --region ap-south-1\n```\n\n### Phase 2: CI/CD Pipeline Configuration\n\n#### 2.1 GitHub Secrets Setup\nNavigate to your repository → Settings → Secrets and variables → Actions\n\n**Required Repository Secrets:**\n- `AWS_ACCESS_KEY_ID`: IAM user access key with EKS/ECR permissions\n- `AWS_SECRET_ACCESS_KEY`: Corresponding secret access key\n- `AWS_REGION`: ap-south-1 (or your chosen region)\n- `AWS_ACCOUNT_ID`: Your 12-digit AWS account identifier\n- `EKS_CLUSTER_NAME`: shopflow-cluster\n\n#### 2.2 Initial Deployment\n```bash\n# Clone and setup repository\ngit clone \u003cyour-repository-url\u003e\ncd shopflow-platform\n\n# Create initial commit and push\ngit add .\ngit commit -m \"feat: initial microservices platform setup\"\ngit push origin main\n\n# Trigger production deployment with version tag\ngit tag v1.0.0\ngit push origin v1.0.0\n```\n\n### Phase 3: Service Mesh Implementation\n\n#### 3.1 Istio Installation\n```bash\n# Download and install Istio 1.27\ncurl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.27.0 sh -\nexport PATH=$PWD/istio-1.27.0/bin:$PATH\n\n# Install Istio with demo profile for learning\nistioctl install --set values.defaultRevision=default -y\n\n# Verify installation\nkubectl get pods -n istio-system\n```\n\n#### 3.2 Service Mesh Integration\n```bash\n# Enable automatic sidecar injection\nkubectl label namespace shopflow-prod istio-injection=enabled\n\n# Restart deployments to inject Envoy sidecars\nkubectl rollout restart deployment -n shopflow-prod\n\n# Verify sidecar injection (should show 2/2 containers)\nkubectl get pods -n shopflow-prod\n```\n\n#### 3.3 Traffic Management Configuration\n```bash\n# Apply Istio Gateway and VirtualService\nkubectl apply -f istio/gateway.yaml\nkubectl apply -f istio/virtualservice.yaml\n\n# Get external access URL\nexport INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')\necho \"Application accessible at: http://$INGRESS_HOST\"\n```\n\n## Project Structure\n\n```\nshopflow-platform/\n├── services/                           # Microservices source code\n│   ├── auth-service/\n│   │   ├── src/\n│   │   ├── package.json\n│   │   └── Dockerfile\n│   ├── product-service/\n│   ├── order-service/\n│   ├── payment-service/\n│   └── notification-service/\n├── k8s/                              # Kubernetes manifests\n│   ├── auth-service.yaml             # Deployment + Service configs\n│   ├── product-service.yaml\n│   ├── order-service.yaml\n│   ├── payment-service.yaml\n│   └── notification-service.yaml\n├── istio/                            # Service mesh configuration\n│   ├── gateway.yaml                  # External traffic entry point\n│   ├── virtualservice.yaml           # Traffic routing rules\n│   └── destinationrules.yaml         # Load balancing policies\n├── .github/workflows/                # CI/CD automation\n│   ├── microservices-ci.yaml         # Development workflow\n│   └── production-deploy.yaml        # Production deployment\n├── docs/                             # Additional documentation\n└── README.md                         # This file\n```\n\n## CI/CD Pipeline Architecture\n\n### Development Workflow\n**Trigger**: Push to `develop` or `main` branches\n```\nBuild Services → Run Tests → Push to ECR → Deploy to Dev → Integration Tests\n```\n\n### Production Workflow\n**Trigger**: Git tag creation (v*.*.*)\n```\nExtract Version → Build All Services → Push to ECR → Deploy to Production → Health Checks\n```\n\n### Pipeline Features\n- **Path-based Service Detection**: Only builds modified services\n- **Parallel Execution**: Concurrent builds for faster deployment\n- **Automated Versioning**: Git tags drive version management\n- **Zero-Downtime Deployment**: Rolling updates with readiness checks\n- **Rollback Capability**: Automated rollback on deployment failure\n\n## Operations \u0026 Management\n\n### Monitoring \u0026 Observability\n```bash\n# Access Kiali dashboard for service mesh visualization\nistioctl dashboard kiali\n\n# Monitor service health\nkubectl get pods -n shopflow-prod -w\n\n# View service logs\nkubectl logs -f deployment/auth-service -n shopflow-prod\n\n# Check service mesh proxy status\nistioctl proxy-status\n```\n\n### Scaling Operations\n```bash\n# Scale individual services\nkubectl scale deployment auth-service --replicas=3 -n shopflow-prod\n\n# Scale cluster nodes (when needed)\neksctl scale nodegroup --cluster shopflow-cluster --name \u003cnodegroup-name\u003e --nodes 3\n```\n\n### Traffic Testing\n```bash\n# Test service endpoints\ncurl -s http://$INGRESS_HOST/products\ncurl -s http://$INGRESS_HOST/orders\ncurl -s http://$INGRESS_HOST/auth/health\n\n# Generate load for testing\nfor i in {1..100}; do\n  curl -s http://$INGRESS_HOST/products \u003e /dev/null\n  sleep 1\ndone\n```\n\n## Challenges Overcome \u0026 Solutions Implemented\n\n### 1. GitHub Actions Secrets Management\n**Challenge**: Initial confusion between environment-scoped and repository-scoped secrets\n**Solution**: Migrated to repository-level secrets for broader accessibility across workflows\n**Learning**: Understanding GitHub Actions secret scoping and environment contexts\n\n### 2. Container Registry Authentication\n**Challenge**: ErrImageNeverPull and ErrImagePull errors during deployment\n**Solution**: \n- Configured proper ECR authentication with `imagePullSecrets`\n- Fixed container image references to use full ECR URIs\n- Implemented ECR login in CI/CD pipeline\n**Learning**: Container registry integration patterns and Kubernetes authentication\n\n### 3. Kubernetes Resource Management\n**Challenge**: Pod scheduling failures due to resource exhaustion\n**Solution**: \n- Scaled EKS node groups to accommodate service mesh overhead\n- Optimized resource requests and limits in deployment manifests\n- Implemented horizontal pod autoscaling readiness\n**Learning**: Production Kubernetes resource planning and capacity management\n\n### 4. Service Mesh Integration Complexity\n**Challenge**: Istio CRD installation failures and sidecar injection issues\n**Solution**:\n- Reinstalled Istio with proper demo profile configuration\n- Correctly labeled namespaces for automatic sidecar injection\n- Configured traffic routing with Gateway and VirtualService resources\n**Learning**: Service mesh architecture patterns and configuration management\n\n### 5. CI/CD Pipeline Optimization\n**Challenge**: Complex multi-service build coordination and deployment sequencing\n**Solution**:\n- Implemented path-based change detection for efficient builds\n- Created environment-specific deployment strategies\n- Added proper error handling and rollback mechanisms\n**Learning**: Advanced GitHub Actions patterns and deployment automation\n\n## Testing \u0026 Validation\n\n### Service Health Verification\n```bash\n# Verify all pods are running with sidecars\nkubectl get pods -n shopflow-prod\n# Expected: Each pod shows 2/2 (application + Envoy proxy)\n\n# Test internal service communication\nkubectl exec -it \u003cpod-name\u003e -n shopflow-prod -- curl http://product-service:4000/health\n\n# Validate external access through Istio Gateway\ncurl -I http://$INGRESS_HOST/products\n# Expected: HTTP 200 OK with Istio headers\n```\n\n### Service Mesh Validation\n```bash\n# Check Istio configuration status\nistioctl analyze -n shopflow-prod\n\n# Verify traffic routing rules\nkubectl get gateway,virtualservice -n shopflow-prod\n\n# Monitor traffic in Kiali dashboard\nistioctl dashboard kiali\n```\n\n## Performance \u0026 Scalability\n\n### Current Capacity\n- **Cluster Nodes**: 2 t3.medium instances (4 vCPU, 8 GB RAM)\n- **Service Replicas**: 1 per service (scalable to 10+)\n- **Request Handling**: ~100 RPS per service with current configuration\n- **Storage**: EBS-backed persistent volumes ready for stateful services\n\n### Scaling Strategies\n- **Horizontal Pod Autoscaling**: Configured based on CPU/memory metrics\n- **Cluster Autoscaling**: Automatic node provisioning based on pod demands\n- **Load Balancing**: Istio-managed traffic distribution across service instances\n\n## Contributing \u0026 Feedback\n\nThis project serves as a comprehensive demonstration of cloud-native engineering practices. While built for educational purposes, the architecture and patterns implemented are production-ready and follow industry best practices.\n\nFor questions about the implementation or discussions about cloud-native architecture patterns, feel free to open an issue in this repository.\n\n## License\n\nThis project is licensed under the MIT License.\n\n## Author\n\n**Shaun Tavitiki**  ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsetavitiki%2Fshopflow-platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsetavitiki%2Fshopflow-platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsetavitiki%2Fshopflow-platform/lists"}