{"id":22508528,"url":"https://github.com/bianchidotdev/sample-scalable-app","last_synced_at":"2026-01-07T06:17:24.163Z","repository":{"id":55580042,"uuid":"322764360","full_name":"bianchidotdev/sample-scalable-app","owner":"bianchidotdev","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-09T03:27:56.000Z","size":170,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-02T02:12:56.671Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bianchidotdev.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-19T04:15:38.000Z","updated_at":"2020-12-23T18:17:37.000Z","dependencies_parsed_at":"2023-01-25T11:01:07.432Z","dependency_job_id":null,"html_url":"https://github.com/bianchidotdev/sample-scalable-app","commit_stats":null,"previous_names":["bianchidotdev/sample-scalable-app"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bianchidotdev%2Fsample-scalable-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bianchidotdev%2Fsample-scalable-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bianchidotdev%2Fsample-scalable-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bianchidotdev%2Fsample-scalable-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bianchidotdev","download_url":"https://codeload.github.com/bianchidotdev/sample-scalable-app/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944105,"owners_count":20697960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T01:21:15.433Z","updated_at":"2026-01-07T06:17:19.134Z","avatar_url":"https://github.com/bianchidotdev.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Express App\n\n## Running the docker image locally\n\n```\n# Create the image\ndocker build -t michaeldbianchi/express-app .\n\n# Run the image\ndocker run -p 8080:3000 -d michaeldbianchi/express-app\n```\n\n## Architecture\n\nWhat's deployed and how:\n\n#### Terraform\n- Networking\n  - VPC\n  - Subnets (public and private)\n  - Route Tables\n  - Internet Gateway\n  - NAT Gateway\n- EKS Cluster\n  - IAM Roles\n- Managed Node Group\n  - Security Groups\n  - CNI Policy\n  - IAM Roles\n- Cluster Autoscaler Role\n\n#### Helm\n\nImperatively applied, but idempotent\n- Cluster Autoscaler\n- Prometheus\n- Grafana\n\n#### Kubernetes Manifests\n\n- Express App\n  - Deployment\n  - Service\n  - Horizontal Pod Autoscaler\n  - Ingress resource (for NGINX)\n- NGINX\n  - NGINX Ingress Controller with NLB automatically created\n- Kubernetes Dashboard (for quick overview of cluster)\n- Metrics Server (to provide pod and node infra metrics)\n\n---\n\n\n### Major Components/Decisions\n\n#### Ingress Controller\n\nUsing NGINX ingress controller to meet simple routing needs. I attempted to use Contour, which has an envoy proxy service, but it wasn't working.\n\nUsing an ingress controller to limit the # of required Load Balancers for the cluster, and also for providing blue-green deployment without DNS cutovers.\n\nA more powerful ingress controller could also provide metrics, tracing, and support for canary style deployments\n\n#### NLB in front of NGINX Ingress Controller\n\nALB worked just fine in front of the app, but not so much in front of NGINX for some reason. \n\nIdeally we would use an ALB either directly created by the ingress controller services in the EKS cluster, or in front of the NLBs created by the cluster in order to provide a WAF.\n\n#### Managed Node Groups\n\nBecause why worry about patching servers for no reason. In all reality, I'd use managed node groups unless there was a really compelling reason to use a custom AMI.\n\n#### eksctl, helm, and kubectl\n\nIn the scope of this exercise, I used a variety of eksctl commands, remote helm charts, and declarative manifests in this repo to set up this cluster.\n\nIn a real production cluster, I would want everything to be declaratively configured. There's no techincal reason I know that we couldn't get there, just not enough time for the exercise.\n\nI configured the deploy such that even though there are imperative commands, the deploy script is idempotent\n\n#### GitHub Actions\n\nUsed as both CI and CD currently, to test/lint the codebase, build and push the docker image, and deploy all the TF and K8s resources\n\nAll of this is fairly fragile currently and would benefit from a more robust CD solution.\n\n\n### Optimizations for another time (in rough order of importance)\n1. HTTPS\n1. Separate repos for Networking (VPC/subnet/NAT), EKS Cluster, and application\n1. Dynamic image tagging - There are rollback challenges when you use a static image tag for deployment\n1. ALB in front of EKS cluster (either through)\n1. Log forwarding to some log aggregator/indexer\n1. Ideally in production, we would have access to resources like Prometheus, Grafana, K8s dashboard through an internal ingress controller accessible only via VPN\n1. Additional observability of networking into applications, providing metrics such as response time, and tracing of each request (solvable using Envoy Sidecars)\n1. Argo CD (or other) for more k8s-native application of manifests\n1. Some namespacing of applications to map to domains/teams\n1. Make the image smaller - likely needs alpine plus multi-stage build\n1. Standard helm chart for all applications to derive from (rather than requiring developers to manage all of their k8s manifests)\n1. Terraform environments with tfvar files for overrides (potentially used for multi-region as well)\n\n\n## Deploying\n\n### Infra/Cluster\n\nSet up aws credentials\n\n```sh\nexport AWS_ACCESS_KEY_ID=\u003caccess_key_id\u003e\nexport AWS_SECRET_ACCESS_KEY=\u003csecret_access_key\u003e\n```\n\nRun `./bin/check-deps.sh \u0026\u0026 ./bin/setup-cluster.sh`\n\nSteps in script:\n1. Run Terraform\n1. Configure kubectl\n1. Apply namespaces (done early so helm charts can be applied into them)\n1. Enable IRSA (required for cluster autoscaler)\n1. Set up necessary values files needed for helm charts (interpolates current AWS Account ID and Cluster Name)\n1. Install helm charts for Cluster Autoscaler, Prometheus, and Grafana  (formerly ALB Controller)\n1. Apply all k8s manifests\n\nExtra:\n\nRight now manually import k8s grafana dashboard. Should be possible from configMap with a little bit of work\n\nClick import, enter 3119 for cluster monitoring and 6417 for pod monitoring\n\n### Application\n\nEither update the code and push to GitHub, which triggers a deploy action or run the deploy app script\n\n```sh\n# expects docker creds, correctly configured kubectl (usually set up during setup-cluster.sh script)\n./bin/deploy-app.sh express-app\n```\n\n### Blue-Green Deploy\n\nUpdate app code as needed, and deploy the code using the following:\n\n```sh\n./bin/deploy-app.sh express-app-blue\n\n# port forward to test new deploy\nkubectl port-forward deploy/express-app-blue 8080:3000\nopen http://localhost:8080\n```\n\nUpdate `iac/manifests/express-app-ingress.yaml` to point to the `express-app-blue` service and run:\n\n```sh\nkubectl apply -f iac/manifests/express-app-ingress.yaml\n```\n\nTo revert, modify the ingress yaml to point back at `express-app` and re apply the manifest\n\n## Monitor the cluster\n### Metrics server\nProvides access to top commands:\n```sh\nkubectl top nodes\nkubectl top pods\n```\n\n### K8s Dashboard\n\nRetrieve a token declared as a secret:\n```sh\nkubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep eks-admin | awk '{print $1}')\n```\n\nProxy the server and open the dashboard\n```sh\nkubectl proxy\n\nopen http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=default\n# Paste the token into the dashboard\n```\n\n### Prometheus\n\n```sh\nkubectl --namespace=prometheus port-forward deploy/prometheus-server 9090\nopen http://localhost:9090\n```\n\n### Grafana\nGet password\n```sh\nkubectl get secret --namespace grafana grafana -o jsonpath=\"{.data.admin-password}\" | base64 --decode ; echo\n```\n\nPort forward and open the dashboard\n```sh\nkubectl port-forward -n grafana deploy/grafana 9091:3000\nopen http://localhost:9091\n```\n\nImport necessary dashboards (3119 - cluster, 6417 - pods)\n\n## Autoscaling\n\nBoth the express app pods and the cluster autoscales horizontally.\n\n### Cluster Autoscaling\nEither remove the autoscaler and scale the deployment, edit it to have a minimum of 10 pods, or run through the load generation from the horizontal pod autoscaling section below\n```sh\n# Option 1:\nkubectl delete -f manifests/express-app-autoscaler.yaml\nkubectl scale deploy/express-app --replicas=20\n\n# Option 2:\nkubectl edit hpa express-app # and change min replicas to 20\n```\n\nThe cpu requests for each pod are artificially too high and will trigger cluster autoscaling prematurely for show\n\n### Horizontal Pod Autoscaling\n```\nkubectl run -it load-generator --image=busybox -- /bin/sh -c 'while true; do wget -q -O - http://express-app/expensive; done'\n\n# Review autoscaler with the following\nkubectl get hpa express-app -w\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbianchidotdev%2Fsample-scalable-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbianchidotdev%2Fsample-scalable-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbianchidotdev%2Fsample-scalable-app/lists"}