{"id":16174202,"url":"https://github.com/relaxdiego/system-design","last_synced_at":"2025-03-19T00:30:57.402Z","repository":{"id":58546936,"uuid":"384655273","full_name":"relaxdiego/system-design","owner":"relaxdiego","description":"Containerized 3-Tier application deployed on AWS' managed kubernetes service.","archived":false,"fork":false,"pushed_at":"2023-02-14T22:17:06.000Z","size":221,"stargazers_count":13,"open_issues_count":1,"forks_count":48,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-15T04:44:12.232Z","etag":null,"topics":["aws","containers","kubernetes","terraform"],"latest_commit_sha":null,"homepage":"https://relaxdiego.com","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/relaxdiego.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-10T09:01:44.000Z","updated_at":"2024-03-27T20:06:13.000Z","dependencies_parsed_at":"2024-10-27T19:20:56.819Z","dependency_job_id":"0e06dd98-9ebe-4893-b8d6-574fcd5f3b3b","html_url":"https://github.com/relaxdiego/system-design","commit_stats":{"total_commits":4,"total_committers":1,"mean_commits":4.0,"dds":0.0,"last_synced_commit":"50f0425ffa20fc37d11a1a2b945ae13548f5d3a5"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relaxdiego%2Fsystem-design","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relaxdiego%2Fsystem-design/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relaxdiego%2Fsystem-design/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/relaxdiego%2Fsystem-design/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/relaxdiego","download_url":"https://codeload.github.com/relaxdiego/system-design/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244331699,"owners_count":20435976,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","containers","kubernetes","terraform"],"created_at":"2024-10-10T04:24:11.223Z","updated_at":"2025-03-19T00:30:57.027Z","avatar_url":"https://github.com/relaxdiego.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# System Design: 3-Tier Web App on AWS EKS\n\nThis is a companion repo for the series of videos I created below. Please\nnote that this codebase is for educational purposes only and is NOT guaranteed\nto be the best implementation for production purposes. For more technical\ndiscussions on Kubernetes, CI/CD, and software engineering in general, please\nvisit [relaxdiego.com](https://relaxdiego.com).\n\n\n## Video Walkthroughs\n\n1. [Part 1](https://youtu.be/8-M2rK4NRyI)\n2. [Part 2](https://youtu.be/RuCOqveUM9k)\n\n\n## Prerequisites\n\n### Set up your AWS client\n\nFirst, ensure that you've configured your AWS CLI accordingly. Setting\nthat up is outside the scope of this guide so please go ahead and read\nup at https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html\n\n\n### Install Terraform\n\nGrab the latest Terraform CLI [here](https://www.terraform.io/downloads.html)\n\n\n### Install kubectl\n\nGrab it via [this guide](https://kubernetes.io/docs/tasks/tools/#kubectl)\n\n\n### Install eksctl\n\nGrab it via [this guide](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)\n\n\n### Install Helm\n\nGrab it via [this guide](https://helm.sh/docs/intro/install/)\n\n\n## Provision the Infrastructure\n\n### Initialize the Terraform Working Directory\n\n```\ncd \u003cPROJECT-ROOT\u003e\n\nterraform -chdir=terraform init\n```\n\n\n### Create Your Environment-Specific tfvars File\n\n```\ncp terraform/example.tfvars terraform/terraform.tfvars\n```\n\nThen modify the file as you see fit.\n\n\n### Create the DB Credentials Secret in AWS\n\n```\nsecrets_dir=~/.relaxdiego/system-design/secrets\nmkdir -p $secrets_dir\nchmod 0700 $secrets_dir\n\naws_profile=$(grep -E ' *aws_profile *=' terraform/terraform.tfvars | sed -E 's/ *aws_profile *= *\"(.*)\"/\\1/g')\naws_region=$(grep -E ' *aws_region *=' terraform/terraform.tfvars | sed -E 's/ *aws_region *= *\"(.*)\"/\\1/g')\ncluster_name=$(grep -E ' *cluster_name *=' terraform/terraform.tfvars | sed -E 's/ *cluster_name *= *\"(.*)\"/\\1/g')\ndb_creds_secret_name=${cluster_name}-db-creds\ndb_creds_secret_file=${secrets_dir}/${cluster_name}-db-creds.json\n\ncat \u003e $db_creds_secret_file \u003c\u003cEOF\n{\n    \"db_user\": \"SU_$(uuidgen | tr -d '-')\",\n    \"db_pass\": \"$(uuidgen)\"\n}\nEOF\nchmod 0600 $db_creds_secret_file\n\naws secretsmanager create-secret \\\n  --profile \"$aws_profile\" \\\n  --name \"$db_creds_secret_name\" \\\n  --description \"DB credentials for ${cluster_name}\" \\\n  --secret-string file://$db_creds_secret_file\n```\n\n\n### Create a Route 53 Zone for Your Environment\n\nFirst, get a hold of an FQDN that you own and define it in an env var:\n\n```\nroute53_zone_fqdn=\u003cTYPE-IN-YOUR-FQDN-HERE\u003e\n```\n\nLet's also create a unique caller reference:\n\n```\nroute53_caller_reference=$(uuidgen | tr -d '-')\n```\n\nThen, create the zone:\n\n```\naws_profile=$(grep -E ' *aws_profile *=' terraform/terraform.tfvars | sed -E 's/ *aws_profile *= *\"(.*)\"/\\1/g')\naws_region=$(grep -E ' *aws_region *=' terraform/terraform.tfvars | sed -E 's/ *aws_region *= *\"(.*)\"/\\1/g')\n\naws route53 create-hosted-zone \\\n  --profile \"$aws_profile\" \\\n  --name \"$route53_zone_fqdn\" \\\n  --caller-reference \"$route53_caller_reference\" \u003e tmp/create-hosted-zone.out\n```\n\nList the nameservers for your zone:\n\n```\ncat tmp/create-hosted-zone.out | jq -r '.DelegationSet.NameServers[]'\n```\n\nNow modify your DNS servers to use the hosts listed above.\n\n\n### Provision the Environment\n\n```\nterraform -chdir=terraform apply\n```\n\nProceed once the above is done.\n\n\n### (Optional) Connect to the Bastion for the First Time\n\nUse [ssh4realz](https://github.com/relaxdiego/ssh4realz) to ensure\nyou connect to the bastion securely. For a guide on how to (and why)\nuse the script, see [this video](https://youtu.be/TcmOd4whPeQ).\n\n```\nssh4realz $(terraform -chdir=terraform output -raw bastion_instance_id)\n```\n\n\n### Subsequent Bastion SSH Connections\n\nWith the bastion's host key already saved to your known_hosts file,\njust SSH directly to its public ip.\n\n```\nssh -A ubuntu@$(terraform -chdir=terraform output -raw bastion_public_ip)\n```\n\n\n### Set-up Your kubectl Config File\n\nBack in your local machine\n\n```\naws eks --region=$(terraform -chdir=terraform output -raw aws_region) \\\n  update-kubeconfig \\\n  --dry-run \\\n  --name $(terraform -chdir=terraform output -raw k8s_cluster_name) \\\n  --alias $(terraform -chdir=terraform output -raw cluster_name) | \\\n  sed -E \"s/^( *(cluster|name)): *arn:.*$/\\1: $(terraform -chdir=terraform output -raw cluster_name)/g\" \\\n  \u003e ~/.kube/config\n\nkubectl config use-context $(terraform -chdir=terraform output -raw cluster_name)\n\nchmod 0600 ~/.kube/config\n```\n\nCheck that you're able to connect to the kube-api-server:\n\n```\nkubectl get pods --all-namespaces\n```\n\n\n### Sanity Check: Test that Pods Can Reach the DB\n\n```\n# Print out the DB endpoint for reference\nterraform -chdir=terraform output db_endpoint\n\nkubectl run -i --tty --rm debug --image=busybox --restart=Never -- sh\n```\n\nOnce in the prompt, run:\n\n```\n/ # telnet \u003cHOSTNAME-PORTION-OF-db_endpoint-OUTPUT\u003e \u003cPORT-PORTION-OF-db_endpoint-OUTPUT\u003e\n```\n\nIt should output:\n\n```\nConnected to \u003cHOSTNAME\u003e\n```\n\nTo exit:\n\n```\n\u003cPress Ctrl-] then Enter then e\u003e\n/ # exit\n```\n\n\n### Log in to the UI and API Container Registries\n\n```\naws ecr get-login-password --region $(terraform -chdir=terraform output -raw aws_region) | \\\n  docker login --username AWS --password-stdin $(terraform -chdir=terraform output -raw registry_frontend)\n\naws ecr get-login-password --region $(terraform -chdir=terraform output -raw aws_region) | \\\n  docker login --username AWS --password-stdin $(terraform -chdir=terraform output -raw registry_api)\n```\n\n\n### Ensure Your Cluster Has an OpenID Connect Provider\n\nOIDC will be used by some pods in the cluster to connect to the AWS API.\nThis section will be based off of [this guide](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html)\n\nFirst check if the cluster already has an OIDC provider:\n\n```\naws eks describe-cluster \\\n    --region $(terraform -chdir=terraform output -raw aws_region) \\\n    --name $(terraform -chdir=terraform output -raw k8s_cluster_name) \\\n    --query \"cluster.identity.oidc.issuer\" \\\n    --output text\n```\n\nIt should return something like:\n\n```\nhttps://oidc.eks.us-west-2.amazonaws.com/id/EXAMPLED539D4633E53DE1B716D3041E\n```\n\nNow grep that sample ID from your list of OIDC providers:\n\n```\naws iam list-open-id-connect-providers | grep \u003cEXAMPLED539D4633E53DE1B716D3041E\u003e\n```\n\nIf the above command returned an ARN, you're done with this section. If\nit did not return one, then run:\n\n```\neksctl utils associate-iam-oidc-provider \\\n    --region $(terraform -chdir=terraform output -raw aws_region) \\\n    --cluster $(terraform -chdir=terraform output -raw k8s_cluster_name) \\\n    --approve\n```\n\nRerun the aws iam command above again (including the pipe to grep) to\ndouble check. It should return a value this time.\n\n\n### Install cert-manager\n\nWhile AWS has its own [certificate management service](https://aws.amazon.com/certificate-manager/),\nwe will work with [cert-manager](https://cert-manager.io) for this exercise\njust so that we'll also learn how to use it. I mean, we're already learning\nstuff, so we might as well!\n\n```\nkubectl apply --validate=false -f apps/cert-manager/cert-manager.yaml\n```\n\nWatch for the status of each cert-manager pod via:\n\n```\nwatch -d kubectl get pods -n cert-manager\n```\n\n\n### Install the Load Balancer Controller\n\nThe AWS LB Controller allows us to create an AWS ALB by simply creating\nIngress resources in our k8s cluster thereby exposing our front end and\nAPI services to the world. We will base the following steps on\n[this guide](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/deploy/installation/)\n\n```\naws iam create-policy \\\n    --policy-name AWSLoadBalancerControllerIAMPolicy \\\n    --policy-document file://apps/aws-lb-controller/iam-policy.json | \\\n  tee tmp/aws-load-balancer-controller-iam-policy.json\n\naws_account_id=$(terraform -chdir=terraform output -raw aws_account_id)\nk8s_cluster_name=$(terraform -chdir=terraform output -raw k8s_cluster_name)\n\neksctl create iamserviceaccount \\\n--cluster=\"$k8s_cluster_name\" \\\n--namespace=kube-system \\\n--name=aws-load-balancer-controller \\\n--attach-policy-arn=arn:aws:iam::${aws_account_id}:policy/AWSLoadBalancerControllerIAMPolicy \\\n--override-existing-serviceaccounts \\\n--approve\n\ncat apps/aws-lb-controller/load-balancer.yaml | \\\n  sed 's@--cluster-name=K8S_CLUSTER_NAME@'\"--cluster-name=${k8s_cluster_name}\"'@' | \\\n  kubectl apply -f -\n```\n\nWatch the aws-load-balancer-controller-xxxxxx-yyyy pod go up via:\n\n```\nwatch -d kubectl get pods -n kube-system\n```\n\n\n### Create the cert-manager Cluster Issuer\n\nThe following steps are based off of [this guide](https://cert-manager.io/docs/configuration/acme/dns01/route53/),\nand [this bit of a (working) hack](https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1084#issuecomment-725566515):\n\nNext, let's deploy the cluster issuer in another terminal:\n\n```\nroute53_zone_fqdn=$(cat tmp/create-hosted-zone.out | jq -r '.HostedZone.Name' | rev | cut -c2- | rev)\nroute53_zone_id=$(cat tmp/create-hosted-zone.out | jq -r '.HostedZone.Id')\ncluster_name=$(terraform -chdir=terraform output -raw cluster_name)\naws_region=$(terraform -chdir=terraform output -raw aws_region)\ncert_manager_role_arn=$(terraform -chdir=terraform output -raw cert_manager_role_arn)\n\ncat apps/cert-manager/cluster-issuer.yaml | \\\n  sed 's@ROUTE53_ZONE_FQDN@'\"${route53_zone_fqdn}\"'@' | \\\n  sed 's@ROUTE53_ZONE_ID@'\"${route53_zone_id}\"'@' | \\\n  sed 's@CLUSTER_NAME@'\"${cluster_name}\"'@' | \\\n  sed 's@AWS_REGION@'\"${aws_region}\"'@' | \\\n  sed 's@CERT_MANAGER_ROLE_ARN@'\"${cert_manager_role_arn}\"'@' | \\\n  kubectl apply -f -\n```\n\nCheck that it created the secret for our app:\n\n```\nkubectl get secret ${cluster_name}-issuer-pkey -n cert-manager\n```\n\n\n### Prepare the App's Namespace\n\n```\nkubectl create ns system-design\n```\n\n### Add the DB Credentials as a Secret\n\nNOTE: This isn't a recommended approach for production environments. If\nyou want something more robus, see [this guide](https://docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_csi_driver.html).\n\n```\ncluster_name=$(terraform -chdir=terraform output -raw cluster_name)\n\nkubectl create secret generic postgres-credentials \\\n  -n system-design \\\n  --from-env-file \u003c(jq -r \"to_entries|map(\\\"\\(.key)=\\(.value|tostring)\\\")|.[]\" ~/.relaxdiego/system-design/secrets/${cluster_name}-db-creds.json)\n```\n\n\n### Wait for App Events\n\nFirst, lets follow events in the system-design namespace to know what's happening\nwhen we apply our manifest later:\n\n```\nkubectl get events -n system-design -w\n```\n\n\n### Build and Deploy the API\n\n```\nmake api\n```\n\nIn the other terminal session where you're watching events, wait for this line:\n\n```\nIssuing   The certificate has been successfully issued\n```\n\nBe patient though as it can take a few minutes and you'll see errors like these:\n\n```\nError presenting challenge: Time limit exceeded. Last error:\nFailed build model due to ingress: system-design/ingress-sysem-design-api: none certificate found for host: api.\u003cfqdn\u003e\n```\n\nIgnore those. Check the status as well via:\n\n```\nhttps://check-your-website.server-daten.de/?q=${component}.${route53_zone_fqdn}\n```\n\n\n### Import the Key and Cert to ACM and Add the API FQDN to Route53\n\nOne downside to using cert-manager with AWS LB Controller is that they don't have\na seamless integration at the time of writing. So once cert-manager is done creating\nthe key and cert, we have to push them to ACM so that ALB can use them.\n\n```\nscripts/sync-tls-resources api\n```\n\nOnce this script completes, the AWS LB Controller will be able to\ncreate the ALB. Next, we update the API DNS record to point to the ALB:\n\n```\nscripts/route53-recordset create api\n```\n\n\n### Build and Deploy the Frontend\n\n```\nmake frontend\n```\n\nIn the other terminal session where you're watching events, wait for this line:\n\n```\nIssuing   The certificate has been successfully issued\n```\n\nBe patient though as it can take a few minutes and you'll see errors like these:\n\n```\nError presenting challenge: Time limit exceeded. Last error:\nFailed build model due to ingress: system-design/ingress-sysem-design-ui: none certificate found for host: ui.\u003cfqdn\u003e\n```\n\nIgnore those. Check the status as well via:\n\n```\nhttps://check-your-website.server-daten.de/?q=${component}.${route53_zone_fqdn}\n```\n\n\n### Import the Key and Cert to ACM and Add the Frontend FQDN to Route53\n\nOne downside to using cert-manager with AWS LB Controller is that they don't have\na seamless integration at the time of writing. So once cert-manager is done creating\nthe key and cert, we have to push them to ACM so that ALB can use them.\n\n```\nscripts/sync-tls-resources ui\n```\n\nOnce this script completes, the AWS LB Controller will be able to\ncreate the ALB. Next, we update the API DNS record to point to the ALB:\n\n```\nscripts/route53-recordset create ui\n```\n\n\n### Clean Up Your Mess!\n\n```\nscripts/route53-recordset delete api\nscripts/route53-recordset delete ui\n\nkubectl delete ns system-design\n\nscripts/delete-tls-resources api\nscripts/delete-tls-resources ui\n\neksctl delete iamserviceaccount \\\n  --cluster=$(terraform -chdir=terraform output -raw k8s_cluster_name) \\\n  --namespace=kube-system \\\n  --name=aws-load-balancer-controller\n\nterraform -chdir=terraform destroy\n\naws iam delete-policy --policy-arn $(cat tmp/aws-load-balancer-controller-iam-policy.json | jq -r '.Policy.Arn')\n```\n\nIf you no longer plan on bringing up this cluster at a later time, clean\nup the following as well:\n\n```\naws_profile=$(grep -E ' *aws_profile *=' terraform/terraform.tfvars | sed -E 's/ *aws_profile *= *\"(.*)\"/\\1/g')\naws_region=$(grep -E ' *aws_region *=' terraform/terraform.tfvars | sed -E 's/ *aws_region *= *\"(.*)\"/\\1/g')\ncluster_name=$(grep -E ' *cluster_name *=' terraform/terraform.tfvars | sed -E 's/ *cluster_name *= *\"(.*)\"/\\1/g')\nroute53_zone_fqdn=$(cat tmp/create-hosted-zone.out | jq -r '.HostedZone.Name')\nroute53_caller_reference=$(uuidgen | tr -d '-')\n\naws secretsmanager delete-secret \\\n  --force-delete-without-recovery \\\n  --secret-id \"${cluster_name}-db-creds\"\n\naws route53 delete-hosted-zone \\\n  --profile \"$aws_profile\" \\\n  --name \"$route53_zone_fqdn\" \\\n  --caller-reference \"$route53_caller_reference\" \u003e tmp/create-hosted-zone.out\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelaxdiego%2Fsystem-design","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frelaxdiego%2Fsystem-design","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelaxdiego%2Fsystem-design/lists"}