{"id":39835039,"url":"https://github.com/muthuri-dev/devops-eks-helm-terraform-ansible","last_synced_at":"2026-01-18T13:20:10.345Z","repository":{"id":317608275,"uuid":"1067808348","full_name":"muthuri-dev/devops-eks-helm-terraform-ansible","owner":"muthuri-dev","description":"Its devops time","archived":false,"fork":false,"pushed_at":"2025-10-01T22:54:15.000Z","size":33,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-02T00:12:15.213Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muthuri-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-01T12:23:09.000Z","updated_at":"2025-10-01T23:10:14.000Z","dependencies_parsed_at":"2025-10-02T00:12:16.582Z","dependency_job_id":"d8cdab26-f5e4-4bed-8c53-8a3c5ef6751f","html_url":"https://github.com/muthuri-dev/devops-eks-helm-terraform-ansible","commit_stats":null,"previous_names":["muthuri-dev/devops-eks-helm-terraform-ansible"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/muthuri-dev/devops-eks-helm-terraform-ansible","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muthuri-dev%2Fdevops-eks-helm-terraform-ansible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muthuri-dev%2Fdevops-eks-helm-terraform-ansible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muthuri-dev%2Fdevops-eks-helm-terraform-ansible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muthuri-dev%2Fdevops-eks-helm-terraform-ansible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muthuri-dev","download_url":"https://codeload.github.com/muthuri-dev/devops-eks-helm-terraform-ansible/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muthuri-dev%2Fdevops-eks-helm-terraform-ansible/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28536751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T13:04:05.990Z","status":"ssl_error","status_checked_at":"2026-01-18T13:01:44.092Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-18T13:20:10.218Z","updated_at":"2026-01-18T13:20:10.324Z","avatar_url":"https://github.com/muthuri-dev.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Production EKS Infrastructure with Terraform, Helm \u0026 GitOps\n\nA production-ready Kubernetes infrastructure on Amazon EKS featuring comprehensive observability, automated CI/CD with GitOps, centralized logging, metrics monitoring, secrets management, and SSL certificate automation.\n\n\u003cimg width=\"2362\" height=\"1507\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/9d50db51-4317-4cf5-b381-a8ae79d19591\" /\u003e\n\n## 🏗️ Architecture Overview\n\n### High-Level Infrastructure Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                                  AWS Cloud                                       │\n│  ┌─────────────────────────────────────────────────────────────────────────────┐ │\n│  │                            VPC (10.0.0.0/16)                                │ │\n│  │                                                                             │ │\n│  │  ┌──────────────────┐                          ┌──────────────────┐         │ │\n│  │  │  Public Subnet   │                          │  Public Subnet   │         │ │\n│  │  │  (us-east-1a)    │                          │  (us-east-1b)    │         │ │\n│  │  │                  │                          │                  │         │ │\n│  │  │  ┌─────────────┐ │                          │ ┌──────────────┐ │         │ │\n│  │  │  │ NAT Gateway │ │                          │ │NGINX Ingress │ │         │ │\n│  │  │  │             │ │                          │ │  Controller  │ │         │ │\n│  │  │  └─────────────┘ │                          │ │  (LoadBalancer)          │ │\n│  │  └──────────────────┘                          │ └──────────────┘ │         │ │\n│  │           │                                     └──────────────────┘        │ │\n│  │           │                                              │                  │ │\n│  │  ┌──────────────────┐                          ┌──────────────────┐         │ │\n│  │  │ Private Subnet   │                          │ Private Subnet   │         │ │\n│  │  │  (us-east-1a)    │                          │  (us-east-1b)    │         │ │\n│  │  │                  │                          │                  │         │ │\n│  │  │ ┌──────────────┐ │                          │ ┌──────────────┐ │         │ │\n│  │  │ │ EKS Workers  │ │                          │ │ EKS Workers  │ │         │ │\n│  │  │ │ (Auto-Scale) │ │                          │ │ (Auto-Scale) │ │         │ │\n│  │  │ │              │ │                          │ │              │ │         │ │\n│  │  │ │ ┌──────────┐ │ │                          │ │ ┌──────────┐ │ │         │ │\n│  │  │ │ │ArgoCD    │ │ │                          │ │ │Prometheus│ │ │         │ │\n│  │  │ │ │Vault     │ │ │                          │ │ │Grafana   │ │ │         │ │\n│  │  │ │ │Kibana    │ │ │                          │ │ │Apps      │ │ │         │ │\n│  │  │ │ │PostgreSQL│ │ │                          │ │ │Fluent Bit│ │ │         │ │\n│  │  │ │ └──────────┘ │ │                          │ │ └──────────┘ │ │         │ │\n│  │  │ └──────────────┘ │                          │ └──────────────┘ │         │ │\n│  │  └──────────────────┘                          └──────────────────┘         │ │\n│  └─────────────────────────────────────────────────────────────────────────────┘ │\n│                                                                                  │\n│  ┌─────────────────────────────────────────────────────────────────────────────┐ │\n│  │                          Amazon ECR (Container Registry)                    │ │\n│  └─────────────────────────────────────────────────────────────────────────────┘ │\n└─────────────────────────────────────────────────────────────────────────────────┘\n\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                              GitHub Repository                                   │\n│  ┌────────────────────┐              ┌──────────────────────┐                   │\n│  │ Application Code   │              │   Helm Charts        │                   │\n│  │  (Dev/Main Branch) │              │  (values.yaml)       │                   │\n│  └────────────────────┘              └──────────────────────┘                   │\n└─────────────────────────────────────────────────────────────────────────────────┘\n```\n\n### Traffic Flow Architecture\n\n```\n                                    Internet User\n                                         │\n                                         │ HTTPS Request\n                                         │ (*.shipcodes.tech)\n                                         ▼\n                              ┌──────────────────────┐\n                              │   Route 53 / DNS     │\n                              │  (Domain Resolution) │\n                              └──────────────────────┘\n                                         │\n                                         │\n                                         ▼\n                              ┌──────────────────────┐\n                              │   AWS Load Balancer  │\n                              │  (Created by NGINX)  │\n                              └──────────────────────┘\n                                         │\n                                         │\n                    ┌────────────────────┼────────────────────┐\n                    │                    │                    │\n                    ▼                    ▼                    ▼\n         ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐\n         │  EKS Worker Node │ │  EKS Worker Node │ │  EKS Worker Node │\n         │                  │ │                  │ │                  │\n         │ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │\n         │ │NGINX Ingress │ │ │ │NGINX Ingress │ │ │ │NGINX Ingress │ │\n         │ │  Controller  │ │ │ │  Controller  │ │ │ │  Controller  │ │\n         │ │   (DaemonSet)│ │ │ │   (DaemonSet)│ │ │ │   (DaemonSet)│ │\n         │ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │\n         └──────────────────┘ └──────────────────┘ └──────────────────┘\n                    │                    │                    │\n                    │ SSL Termination    │                    │\n                    │ (Let's Encrypt)    │                    │\n                    └────────────────────┼────────────────────┘\n                                         │\n                              ┌──────────▼──────────┐\n                              │  Ingress Resources  │\n                              │  (Route by Domain)  │\n                              └─────────────────────┘\n                                         │\n              ┌──────────────────────────┼──────────────────────────┐\n              │                          │                          │\n              ▼                          ▼                          ▼\n    ┌──────────────────┐      ┌──────────────────┐      ┌──────────────────┐\n    │ ArgoCD Service   │      │ Grafana Service  │      │  Vault Service   │\n    │ (ClusterIP)      │      │ (ClusterIP)      │      │  (ClusterIP)     │\n    └──────────────────┘      └──────────────────┘\n              │                          │                          │\n              ▼                          ▼                          ▼\n    ┌──────────────────┐      ┌──────────────────┐      ┌──────────────────┐\n    │  ArgoCD Pods     │      │  Grafana Pods    │      │   Vault Pods     │\n    │  (Deployment)    │      │  (StatefulSet)   │      │  (StatefulSet)   │\n    └──────────────────┘      └──────────────────┘      └──────────────────┘\n```\n\n\u003cimg width=\"1181\" height=\"2340\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/148eced5-e2d7-4454-9f52-4d52a961aaa9\" /\u003e\n\n### Request/Response Flow Detail\n\n```\n1. User Request:\n   https://argocd.shipcodes.tech\n          │\n          ▼\n2. DNS Resolution:\n   Cloudflare → AWS Load Balancer IP\n          │\n          ▼\n3. Load Balancer:\n   Distributes to NGINX Ingress on any worker node\n          │\n          ▼\n4. NGINX Ingress Controller:\n   - Reads Ingress resource\n   - Checks host: argocd.shipcodes.tech\n   - Terminates SSL (Let's Encrypt cert)\n   - Routes to backend service\n          │\n          ▼\n5. Kubernetes Service:\n   argocd-server (ClusterIP) on port 80\n          │\n          ▼\n6. Pod Selection:\n   Service selects healthy ArgoCD pod via label selector\n          │\n          ▼\n7. ArgoCD Pod:\n   Processes request and generates response\n          │\n          ▼\n8. Response Path (Reverse):\n   Pod → Service → Ingress → Load Balancer → User\n```\n\n## 🎯 Complete Infrastructure Components\n\n### ✅ Core Kubernetes Infrastructure\n\n- **Amazon EKS Cluster**: Managed Kubernetes control plane\n- **EKS Node Groups**: Auto-scaling worker nodes (2-12 nodes)\n- **Cluster Autoscaler**: Automatic node scaling based on pod demand\n- **VPC \u0026 Networking**: Production-grade multi-AZ setup\n- **EBS CSI Driver**: Persistent storage with dynamic provisioning\n- **Metrics Server**: Resource metrics for HPA and monitoring\n\n### ✅ Ingress \u0026 Networking\n\n- **NGINX Ingress Controller**: Centralized ingress with SSL termination\n- **Cert-Manager**: Automatic SSL certificate provisioning via Let's Encrypt\n- **DNS Integration**: Domain-based routing (\\*.shipcodes.tech)\n- **Load Balancer**: AWS NLB/ALB for external traffic\n\n### ✅ Observability Stack\n\n#### Logging (EFK Stack)\n\n- **Elasticsearch**: Centralized log storage and indexing\n- **Fluent Bit**: Lightweight log collector (DaemonSet on all nodes)\n- **Kibana**: Log visualization and analysis dashboard\n\n#### Monitoring\n\n- **Prometheus**: Metrics collection and storage\n- **Grafana**: Metrics visualization and alerting\n- **Service Discovery**: Automatic scraping of Kubernetes metrics\n\n### ✅ GitOps \u0026 CI/CD\n\n- **ArgoCD**: GitOps continuous deployment\n- **GitHub Actions**: CI pipeline for build and push\n- **Helm Charts**: Application packaging and versioning\n\n### ✅ Secrets \u0026 Database\n\n- **HashiCorp Vault**: Centralized secrets management\n- **CloudNativePG**: PostgreSQL operator for database workloads\n- **Amazon ECR**: Private container registry\n\n## 📁 Project Structure\n\n```\ndevops-eks-infrastructure/\n├── infrastructure/                    # Terraform IaC\n│   ├── 01-provider.tf                # Provider configuration\n│   ├── 02-backend.tf                 # S3 backend with state\n│   ├── 03-variables.tf               # Variable definitions\n│   ├── 04-vpc-networking.tf          # VPC, subnets, NAT\n│   ├── 05-eks-cluster.tf             # EKS cluster\n│   ├── 06-ebs-csi.tf                 # EBS CSI driver\n│   ├── 07-nodegroup.tf               # Auto-scaling node groups\n│   ├── 08-monitoring.tf              # Prometheus \u0026 Grafana\n│   ├── 09-argocd.tf                  # ArgoCD with Ingress\n│   ├── 10-logging.tf                 # EFK stack\n│   ├── 11-nginx-ingress.tf           # NGINX Ingress Controller\n│   ├── 12-cert-manager.tf            # Cert-Manager\n│   ├── 13-vault.tf                   # Vault with Ingress\n│   ├── 14-ecr.tf                     # Amazon ECR\n│   ├── 15-cloudnative-pg.tf          # PostgreSQL operator\n│   ├── 16-cluster-autoscaler.tf      # Cluster autoscaler\n│   ├── 17-metrics-server.tf          # Metrics server\n│   └── 18-outputs.tf                 # Infrastructure outputs\n├── helm-charts/                      # Application Helm charts\n│   └── myapp/\n│       ├── Chart.yaml\n│       ├── values.yaml               # Updated by CI/CD\n│       └── templates/\n├── application/                      # Application source code\n│   ├── src/\n│   ├── Dockerfile\n│   └── requirements.txt\n├── .github/\n│   └── workflows/\n│       └── deploy.yml               # GitHub Actions workflow\n└── README.md\n```\n\n## 🔄 Complete CI/CD Pipeline Flow\n\n\u003cimg width=\"2673\" height=\"965\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/2ea5b0a1-e50b-422e-8611-f6cd6872b1cf\" /\u003e\n\n### Pipeline Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                           Developer Workflow                                    │\n└─────────────────────────────────────────────────────────────────────────────────┘\n                                         │\n                                         ▼\n                              ┌──────────────────────┐\n                              │  1. Developer Push   │\n                              │  to 'dev' branch     │\n                              └──────────────────────┘\n                                         │\n                                         ▼\n                              ┌──────────────────────┐\n                              │  2. Create PR        │\n                              │  dev → main          │\n                              └──────────────────────┘\n                                         │\n                                         ▼\n                              ┌──────────────────────┐\n                              │  3. Code Review \u0026    │\n                              │  Merge to main       │\n                              └──────────────────────┘\n                                         │\n                                         ▼\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                         GitHub Actions Workflow                                  │\n├─────────────────────────────────────────────────────────────────────────────────┤\n│                                                                                 │\n│  Step 1: Checkout Code                                                          │\n│  ├─ actions/checkout@v3                                                         │\n│  └─ Fetches repository code                                                     │\n│                                                                                 │\n│  Step 2: Authenticate with Vault                                                │\n│  ├─ Connect to Vault (vault.shipcodes.tech)                                     │\n│  ├─ Retrieve AWS credentials                                                    │\n│  └─ Get ECR registry details                                                    │\n│                │                                                                │\n│                ▼                                                                │\n│         ┌────────────┐                                                          │\n│         │   Vault    │                                                          │\n│         │  Secrets:  │                                                          │\n│         │  - AWS_KEY │                                                          │\n│         │  - AWS_SEC │                                                          │\n│         │  - ECR_URI │                                                          │\n│         └────────────┘                                                          │\n│                │                                                                │\n│  Step 3: Build Docker Image                                                     │\n│  ├─ docker build -t myapp:$GITHUB_SHA                                           │\n│  └─ Tag with commit SHA for versioning                                          │\n│                                                                                 │\n│  Step 4: Push to Amazon ECR                                                     │\n│  ├─ aws ecr get-login-password                                                  │\n│  ├─ docker tag myapp:$SHA $ECR_URI/myapp:$SHA                                   │\n│  └─ docker push $ECR_URI/myapp:$SHA                                             │\n│                │                                                                │\n│                ▼                                                                │\n│         ┌────────────┐                                                          │\n│         │ Amazon ECR │                                                          │\n│         │   Image:   │                                                          │\n│         │ myapp:abc1 │                                                          │\n│         └────────────┘                                                          │\n│                │                                                                │\n│  Step 5: Update Helm Chart                                                      │\n│  ├─ Checkout helm-charts repository                                             │\n│  ├─ Update values.yaml with new image tag                                       │\n│  │  image:                                                                      │\n│  │    repository: $ECR_URI/myapp                                                │\n│  │    tag: abc123def456  # New commit SHA                                       │\n│  ├─ git commit -m \"Update image to abc123\"                                      │\n│  └─ git push to helm-charts repo                                                │\n│                                                                                 |\n└─────────────────────────────────────────────────────────────────────────────────┘\n                                         │\n                                         ▼\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                           ArgoCD GitOps Sync                                     │\n├─────────────────────────────────────────────────────────────────────────────────┤\n│                                                                                 │\n│  Step 1: Detect Changes                                                         │\n│  ├─ ArgoCD monitors helm-charts repository                                      │\n│  ├─ Detects values.yaml change                                                  │\n│  └─ Triggers sync (auto or manual)                                              │\n│                                                                                 │\n│  Step 2: Sync Application                                                       │\n│  ├─ Renders Helm chart with new values                                          │\n│  ├─ Compares with cluster state                                                 │\n│  └─ Applies changes to EKS cluster                                              │\n│                                                                                 │\n│  Step 3: Rolling Update                                                         │\n│  ├─ Kubernetes Deployment rollout                                               │\n│  ├─ Pull new image from ECR                                                     │\n│  ├─ Create new pods with new image                                              │\n│  ├─ Wait for health checks                                                      │\n│  └─ Terminate old pods                                                          │\n│                                                                                 │\n│  Step 4: Health Verification                                                    │\n│  ├─ Check pod readiness probes                                                  │\n│  ├─ Verify service endpoints                                                    │\n│  └─ Application accessible via ingress                                          │\n│                                                                                 |\n└─────────────────────────────────────────────────────────────────────────────────┘\n                                         │\n                                         ▼\n                              ┌──────────────────────┐\n                              │  Application Live!   │\n                              │  https://api         │\n                              │  .shipcodes.tech     │\n                              └──────────────────────┘\n```\n\n\u003cimg width=\"2255\" height=\"1439\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/aa1257e0-81a5-4ca2-aa56-f37043744963\" /\u003e\n\n## 📊 EFK Stack Architecture (Logging)\n\n### EFK Stack Components\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                            EFK Logging Stack                                     │\n└─────────────────────────────────────────────────────────────────────────────────┘\n\n┌──────────────────┐  ┌──────────────────┐  ┌──────────────────┐\n│  EKS Node 1      │  │  EKS Node 2      │  │  EKS Node 3      │\n│                  │  │                  │  │                  │\n│ ┌──────────────┐ │  │ ┌──────────────┐ │  │ ┌──────────────┐ │\n│ │ Fluent Bit   │ │  │ │ Fluent Bit   │ │  │ │ Fluent Bit   │ │\n│ │ (DaemonSet)  │ │  │ │ (DaemonSet)  │ │  │ │ (DaemonSet)  │ │\n│ └──────┬───────┘ │  │ └──────┬───────┘ │  │ └──────┬───────┘ │\n│        │         │  │        │         │  │        │         │\n│ ┌──────▼───────┐ │  │ ┌──────▼───────┐ │  │ ┌──────▼───────┐ │\n│ │  App Pod 1   │ │  │ │  App Pod 3   │ │  │ │  App Pod 5   │ │\n│ │  logs/*.log  │ │  │ │  logs/*.log  │ │  │ │  logs/*.log  │ │\n│ └──────────────┘ │  │ └──────────────┘ │  │ └──────────────┘ │\n│ ┌──────────────┐ │  │ ┌──────────────┐ │  │ ┌──────────────┐ │\n│ │  App Pod 2   │ │  │ │  App Pod 4   │ │  │ │  App Pod 6   │ │\n│ │  logs/*.log  │ │  │ │  logs/*.log  │ │  │ │  logs/*.log  │ │\n│ └──────────────┘ │  │ └──────────────┘ │  │ └──────────────┘ │\n└────────┬─────────┘  └────────┬─────────┘  └────────┬─────────┘\n         │                     │                     │\n         │  Parse \u0026 Forward    │                     │\n         └─────────────────────┼─────────────────────┘\n                               │\n                               ▼\n                    ┌─────────────────────┐\n                    │   Elasticsearch     │\n                    │   (StatefulSet)     │\n                    │                     │\n                    │  • Index: fluent-*  │\n                    │  • Storage: 5GB     │\n                    │  • Replicas: 1      │\n                    └─────────────────────┘\n                               │\n                               │ Query \u0026 Visualize\n                               ▼\n                    ┌─────────────────────┐\n                    │      Kibana         │\n                    │   (Deployment)      │\n                    │                     │\n                    │  kibana.shipcodes   │\n                    │       .tech         │\n                    └─────────────────────┘\n                               │\n                               ▼\n                    ┌─────────────────────┐\n                    │   NGINX Ingress     │\n                    │   (SSL Enabled)     │\n                    └─────────────────────┘\n```\n\n\u003cimg width=\"2255\" height=\"1436\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/f3b7b179-3f5c-4710-ac6b-7d00270f0563\" /\u003e\n\n### Fluent Bit Log Collection Process\n\n```\n1. Fluent Bit DaemonSet:\n   - Deployed on EVERY node in the cluster\n   - Runs as privileged pod with host access\n   - Mounts /var/log/containers from host\n\n2. Log Collection:\n   - Reads container logs: /var/log/containers/*.log\n   - Parses JSON format from container runtime\n   - Extracts metadata: pod, namespace, container name\n\n3. Log Processing:\n   - Filters: Remove system logs if needed\n   - Parsers: JSON, regex for custom formats\n   - Enrichment: Add Kubernetes metadata\n\n4. Log Forwarding:\n   - Protocol: HTTP/HTTPS\n   - Destination: Elasticsearch service\n   - Index: kubernetes-\n   - Buffering: Local disk for reliability\n\n5. Elasticsearch Storage:\n   - Creates daily indices\n   - Applies mapping for log fields\n   - Stores with retention policy\n```\n\n### Kibana Log Visualization Setup\n\n1. Access Kibana at `https://kibana.shipcodes.tech`\n2. Navigate to **Management** → **Index Patterns**\n3. Create index pattern: `kubernetes*`\n4. Select time field: `@timestamp`\n5. Go to **Discover** to view logs\n6. Create visualizations and dashboards\n\n## 📈 Prometheus \u0026 Grafana Monitoring\n\n\u003cimg width=\"1261\" height=\"1500\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/fd269e39-d36e-4557-901b-24785c9d579d\" /\u003e\n\n### Monitoring Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                        Prometheus Monitoring Stack                               │\n└─────────────────────────────────────────────────────────────────────────────────┘\n\n┌──────────────────────────────────────────────────────────────────────────────────┐\n│                            Metrics Sources                                       │\n├──────────────────────────────────────────────────────────────────────────────────┤\n│                                                                                  │\n│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐            │\n│  │ Node Exporter│  │kube-state   │  │  cAdvisor   │  │   Custom    │            │\n│  │             │  │  -metrics   │  │             │  │ App Metrics │            │\n│  │ :9100/metrics  │ :8080/metrics  │ :10250/metrics │ :8080/metrics            │\n│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘            │\n│         │                 │                 │                 │                 │\n│         │                 │                 │                 │                 │\n│         └─────────────────┴─────────────────┴─────────────────┘                 │\n│                                     │                                           │\n│                           Service Discovery                                     │\n│                      (Kubernetes API Integration)                               │\n└──────────────────────────────────────┬───────────────────────────────────────────┘\n                                       │\n                                       │ Scrape every 30s\n                                       ▼\n                            ┌─────────────────────┐\n                            │    Prometheus       │\n                            │   (StatefulSet)     │\n                            │                     │\n                            │  • TSDB Storage     │\n                            │  • Retention: 15d   │\n                            │  • PVC: 20GB        │\n                            │  • HA: Replicas     │\n                            └─────────────────────┘\n                                       │\n                                       │ PromQL Queries\n                                       ▼\n                            ┌─────────────────────┐\n                            │      Grafana        │\n                            │   (StatefulSet)     │\n                            │                     │\n                            │  grafana.shipcodes  │\n                            │       .tech         │\n                            │                     │\n                            │  • Dashboards       │\n                            │  • Alerts           │\n                            │  • PVC: 10GB        │\n                            └─────────────────────┘\n                                       │\n                                       ▼\n                            ┌─────────────────────┐\n                            │   NGINX Ingress     │\n                            │   (SSL Enabled)     │\n                            └─────────────────────┘\n```\n\n\u003cimg width=\"2253\" height=\"1438\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/877262ce-45ac-4ce0-9a34-959fac6effe1\" /\u003e\n\n### Key Metrics Collected\n\n**Node Metrics (Node Exporter)**\n\n- CPU usage per core\n- Memory usage and available\n- Disk I/O and space\n- Network traffic\n\n**Cluster Metrics (kube-state-metrics)**\n\n- Pod status and restarts\n- Deployment replicas\n- Node status\n- Resource requests/limits\n\n**Container Metrics (cAdvisor)**\n\n- Container CPU usage\n- Container memory usage\n- Container network I/O\n- Container filesystem usage\n\n### Grafana Dashboard Access\n\n1. Access Grafana at `https://grafana.shipcodes.tech`\n2. Login with configured credentials\n3. Pre-configured dashboards:\n   - **Kubernetes Cluster Monitoring**: Overall cluster health\n   - **Node Exporter Full**: Detailed node metrics\n   - **Pod Monitoring**: Per-pod resource usage\n   - **Namespace Monitoring**: Resource usage by namespace\n\n## 🔐 Security \u0026 Secrets Management\n\n\u003cimg width=\"2255\" height=\"1444\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/c8a5579b-34fe-4005-b211-db9a02ab8c6a\" /\u003e\n\n### Vault Integration\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│ Vault Secrets Management │\n└─────────────────────────────────────────────────────────────────────────────────┘\n\n                            ┌─────────────────────┐\n                            │   HashiCorp Vault   │\n                            │   (StatefulSet)     │\n                            │                     │\n                            │  vault.shipcodes    │\n                            │       .tech         │\n                            └─────────────────────┘\n                                       │\n                    ┌──────────────────┼──────────────────┐\n                    │                  │                  │\n                    ▼                  ▼                  ▼\n         ┌──────────────────┐ ┌──────────────┐ ┌──────────────────┐\n         │ GitHub Actions   │ │ Application  │ │   Operators      │\n         │                  │ │   Pods       │ │                  │\n         │ • AWS Creds      │ │ • DB Creds   │ │ • API Keys       │\n         │ • ECR Access     │ │ • API Keys   │ │ • Certificates   │\n         │ • Deploy Keys    │ │ • Configs    │ │ • Tokens         │\n         └──────────────────┘ └──────────────┘ └──────────────────┘\n```\n\n## SSL Certificate Management\n\n```\n┌─────────────────────────────────────────────────────────────────────────────────┐\n│                    Cert-Manager Certificate Flow                                 │\n└─────────────────────────────────────────────────────────────────────────────────┘\n\n                              ┌──────────────────────┐\n                              │  Kubernetes Ingress  │\n                              │  Created with TLS    │\n                              │                      │\n                              │  annotations:        │\n                              │   cert-manager.io/   │\n                              │   cluster-issuer:    │\n                              │   letsencrypt-prod   │\n                              └──────────────────────┘\n                                         │\n                                         │ Triggers\n                                         ▼\n                              ┌──────────────────────┐\n                              │   Cert-Manager       │\n                              │   (Deployment)       │\n                              └──────────────────────┘\n                                         │\n                                         │ Creates\n                                         ▼\n                              ┌──────────────────────┐\n                              │  Certificate Object  │\n                              │  (CRD)               │\n                              └──────────────────────┘\n                                         │\n                                         │ ACME Challenge\n                                         ▼\n                              ┌──────────────────────┐\n                              │   Let's Encrypt CA   │\n                              │   (HTTP-01)          │\n                              └──────────────────────┘\n                                         │\n                                         │ Validates domain\n                                         │ Issues certificate\n                                         ▼\n                              ┌──────────────────────┐\n                              │  Kubernetes Secret   │\n                              │  (TLS Certificate)   │\n                              │                      │\n                              │  - tls.crt           │\n                              │  - tls.key           │\n                              └──────────────────────┘\n                                         │\n                                         │ Mounted by\n                                         ▼\n                              ┌──────────────────────┐\n                              │  NGINX Ingress       │\n                              │  (SSL Termination)   │\n                              └──────────────────────┘\n```\n\n## 🚀 Getting Started\n\nPrerequisites\nInstall required tools:\n\n```\n# Terraform\ncurl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -\nsudo apt-add-repository \"deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main\"\nsudo apt-get update \u0026\u0026 sudo apt-get install terraform\n```\n\n# AWS CLI\n\ncurl \"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip\" -o \"awscliv2.zip\"\nunzip awscliv2.zip\nsudo ./aws/install\n\n# kubectl\n\ncurl -LO \"https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl\"\nsudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl\n\n# Helm\n\n```\ncurl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash\nAWS Configuration\nbash# Configure AWS credentials\naws configure\n\n# AWS Access Key ID: YOUR_ACCESS_KEY\n\n# AWS Secret Access Key: YOUR_SECRET_KEY\n\n# Default region: us-east-1\n\n# Default output format: json\n\n# Verify configuration\n```\n\naws sts get-caller-identity\n\n## 🏗️ Infrastructure Deployment\n\n### Step 1: Initialize Terraform\n\n```\ncd infrastructure/\n```\n\n### Initialize Terraform with backend\n\n```\nterraform init\n```\n\n### Validate configuration\n\n```\nterraform validate\n```\n\n### Format configuration files\n\n```\nterraform fmt\n```\n\n### Step 2: Plan Infrastructure\n\n```\n# Review planned changes\nterraform plan\n```\n\n# Save plan to file (optional)\n\n```\nterraform plan\n```\n\n### Step 3: Deploy Infrastructure\n\n```\n# Apply configuration\nterraform apply\n```\n\n# Or apply saved plan\n\nterraform apply\nDeployment creates:\n\n✅ EKS cluster with control plane\n\n✅ VPC with public/private subnets\n\n✅ NAT Gateway for private subnet internet access\n\n✅ Auto-scaling node groups (2-12 nodes)\n\n✅ NGINX Ingress Controller with LoadBalancer\n\n✅ Cert-Manager with Let's Encrypt integration\n\n✅ Prometheus \u0026 Grafana with persistent storage\n\n✅ Elasticsearch, Kibana, and Fluent Bit\n\n✅ ArgoCD with GitOps configuration\n\n✅ HashiCorp Vault for secrets\n\n✅ CloudNativePG PostgreSQL operator\n\n✅ Amazon ECR repository\n\n✅ Cluster Autoscaler\n\n✅ Metrics Server\n\n### Step 4: Configure kubectl\n\n```\nbash# Update kubeconfig for EKS cluster\naws eks update-kubeconfig --region us-east-1 --name production_eks\n```\n\n# Verify cluster access\n\n```\nkubectl cluster-info\n```\n\n# Check all nodes are ready\n\n```\nkubectl get nodes\n```\n\n# View all pods across namespaces\n\n```\nkubectl get pods --all-namespaces\n🌐 Accessing Services\nGet Service URLs\nbash# Get all ingress URLs\nkubectl get ingress --all-namespaces\n\n\n# Expected output:\n\n# NAMESPACE NAME HOSTS ADDRESS\n\n# argocd argocd-ingress argocd.shipcodes.tech \u003cLoadBalancer-DNS\u003e\n\n# monitoring grafana-ingress grafana.shipcodes.tech \u003cLoadBalancer-DNS\u003e\n\n# elastic-stack kibana-ingress kibana.shipcodes.tech \u003cLoadBalancer-DNS\u003e\n\n# vault vault-ingress vault.shipcodes.tech \u003cLoadBalancer-DNS\u003e\n```\n\nConfigure DNS Records\nFor each service, create DNS A/CNAME records pointing to the LoadBalancer:\nbash# Get LoadBalancer DNS\nkubectl get service -n ingress-nginx ingress-nginx-controller\n\nCreate DNS records in Route 53 or your DNS provider like cloudflare:\n\nargocd.shipcodes.tech → LoadBalancer DNS\ngrafana.shipcodes.tech → LoadBalancer DNS\nkibana.shipcodes.tech → LoadBalancer DNS\nvault.shipcodes.tech → LoadBalancer DNS\n\n## 📦 Deploying Applications\n\nApplication Deployment Flow\n\n```\nDeveloper → GitHub (dev) → PR → Merge (main) → GitHub Actions\n↓\n┌───────────┴────────────┐\n│ │\nVault Secrets Build Image\n│ │\n└───────────┬────────────┘\n↓\nPush to ECR\n↓\nUpdate Helm Chart (values.yaml)\n↓\nArgoCD Detects\n↓\nSync to EKS Cluster\n↓\nApplication Deployed\n↓\nAccessible via Ingress\nCreate Application Helm Chart\n```\n\n## 📝 Best Practices\n\nSecurity\n\nRotate Secrets Regularly: Update credentials in Vault periodically\nUse RBAC: Implement least-privilege access controls\nEnable Pod Security: Use Pod Security Standards\nNetwork Policies: Restrict pod-to-pod communication\nImage Scanning: Enable ECR image scanning\nAudit Logging: Enable EKS control plane logging\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuthuri-dev%2Fdevops-eks-helm-terraform-ansible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuthuri-dev%2Fdevops-eks-helm-terraform-ansible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuthuri-dev%2Fdevops-eks-helm-terraform-ansible/lists"}