{"id":23780901,"url":"https://github.com/ajithvcoder/emlo4-session-14-ajithvcoder","last_synced_at":"2026-04-08T22:31:18.055Z","repository":{"id":270326267,"uuid":"909960286","full_name":"ajithvcoder/emlo4-session-14-ajithvcoder","owner":"ajithvcoder","description":"NextJS App + Fastapi Serve + Redis + kubernetes + Helm charts + dog and cat classifier - The School of AI EMLO-V4 course assignment https://theschoolof.ai/#programs","archived":false,"fork":false,"pushed_at":"2024-12-31T13:01:46.000Z","size":1930,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-02T12:43:32.027Z","etag":null,"topics":["classification","fastapi","helm-charts","kubernetes","machine-learning","minikube","redis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajithvcoder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-30T06:44:47.000Z","updated_at":"2025-01-16T01:25:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd2d397e-dc8c-4821-ab5c-c3c73dfedecf","html_url":"https://github.com/ajithvcoder/emlo4-session-14-ajithvcoder","commit_stats":null,"previous_names":["ajithvcoder/emlo4-session-14-ajithvcoder"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ajithvcoder/emlo4-session-14-ajithvcoder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Femlo4-session-14-ajithvcoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Femlo4-session-14-ajithvcoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Femlo4-session-14-ajithvcoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Femlo4-session-14-ajithvcoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajithvcoder","download_url":"https://codeload.github.com/ajithvcoder/emlo4-session-14-ajithvcoder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajithvcoder%2Femlo4-session-14-ajithvcoder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31577444,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","fastapi","helm-charts","kubernetes","machine-learning","minikube","redis"],"created_at":"2025-01-01T11:18:15.826Z","updated_at":"2026-04-08T22:31:18.050Z","avatar_url":"https://github.com/ajithvcoder.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## EMLOV4-Session-14 Assignment - Kubernetes - II: Ingress, ConfigMap, Secrets, Volumes and HELM\n\nNote: **I have completed actual assignment and both the bonus assignment**\n\nUsing Helm charts, we are deploying a `cat-dog model service` hosted on a FastAPI server. Alongside, a `backend service built with FastAPI` handles requests. The `NextJS UI service` functions as the front-end interface for users. Additionally, a `Redis caching service` is integrated to enhance performance. Finally its exposed to internet with `ngrok`\n\n### Contents\n\n- [Requirements](#requirements)\n- [Development Method](#development-method)\n    - [Architecture Diagram](#architecture-diagram)\n    - [Docker file development-Model-service development-Web-service development](#docker-file-development-model-service-development-web-service-development)\n    - [Minikube development](#minikube-development)\n    - [Deploy with HELM](#deploy-with-helm)\n    - [Bonus assignment 1](#bonus-assignment-1)\n    - [Bonus assignment 2](#bonus-assignment-2)\n- [Learnings](#learnings)\n- [Results Screenshots](#results-screenshots)\n\n### Requirements\n\n- Design a deployment to deploy the Cat/Dog or Dog Classifier on K8S\n    - You must first create a architecture diagram and show all pods, replicasets, deployments, service, ingress, volumes and nodes that will be involved in this deployment\n    - Use diagrams.netLinks to an external site. for the diagram\n    - write the code for model server, web server and test it with docker-compose first\n    - you must be using a redis cache for inference results caching\n    - create k8s manifests for the same\n    - You’ll need to figure out the peak cpu and memory usage with kubectl top pod\n- Deploy on K8S using the manifests\n- Create HELM Chart with configurable values\n- Deploy with HELM\n- You must use minikube either on your local machine or on an EC2 instance\n- What to Submit?\n    - Github Repo with Deployment YAML Files\n    1. Instructions to\n        - Deploy using HELM Chart\n        - Tunnel to the Ingress\n        - Screenshot of the fastapi docs page with one inference done\n    2. Output of the following in a .md file in your repository\n        - kubectl describe \u003cyour_deployment\u003e\n        - kubectl describe \u003cyour_pod\u003e\n        - kubectl describe \u003cyour_ingress\u003e\n        - kubectl top pod\n        - kubectl top node\n        - kubectl get all -A -o yaml\n\n**BONUS**\n\n- Use ngrok ingress to expose your deployment to internet\n- Deploy a basic frontend in Next.JS to do model inferencing\n\n### Architecture Diagram\n\n![](./assets/snap_architecture.png)\n\nNote:\n\n- You can refer dev/docker-dev for docker development related testings\n\n- You can refer dev/minikube-dev for minikube development and manifest file related testings. No helm charts there\n\n### Docker file development-Model-service development-Web-service development\n\n**Build**\n\n- `docker build -t model-server -f Dockerfile.model-server .`\n- `docker build -t web-server -f Dockerfile.web-server .`\n- `docker build -t ui-server -f Dockerfile.ui-server .`\n- `docker network create my_network`\n\nDocker network is needed for communication between two containers\n\n**Start Servers**\n\nModel server\n\nNote:\n\n1. I am downloading model inside docker file so dont mount anything in model-server else download in local also and then mount.\n2. Modify the server port while testing as its hosted in 80 by default for kubernetes service deployment\n\n- `docker run -it --network my_network -v /home/ajith/mlops/course/emlo_play/emlo4-s14/emlo4-session-14-ajithvcoder/src/model-server:/opt/src -p8000:8000 model-server bash`\n- `python server.py`\n\nWeb server\n\n- `docker run -it --network my_network  -v /home/ajith/mlops/course/emlo_play/emlo4-s14/emlo4-session-14-ajithvcoder/src/web-server:/opt/src -p9000:9000 web-server bash`\n- `python server.py`\n\nUI server\n\n- `docker run -it --network my_network -v /home/ajith/mlops/course/emlo_play/emlo4-s14/emlo4-session-14-ajithvcoder/src/ui-server/ui:/opt/src -p3000:3000 web-server bash`\n- `npm run dev`\n\nNote: Change the port configuration in source code and expose them properly\n\n**Testing both services**\n\nNote: remember @ before the filepath\n\n- `curl -X POST http://localhost:8000/infer -H \"Content-Type: multipart/form-data\" -F \"image=@dog.jpg\"`\n\n- `curl -X POST http://localhost:9000/classify-catdog -H \"Content-Type: multipart/form-data\" -F \"image=@dog.jpg\"`\n\n    ```\n    {\"class\": dog, \"confidence\": 0.9892888878}\n    ```\n- Open `http://localhost:3000` in UI for UI server testing\n\n- After testing this you can proceed with `minikube`\n\n### Minikube development\n\n- `minikube start`\n\nAdd alias for easier cli work\n\n- `alias kubectl=\"minikube kubectl --\"`\n\nKubernetes environment build docker images\n\n- `eval $(minikube docker-env)`\n- `docker build --platform linux/amd64 -t model-server -f Dockerfile.model-server  .`\n- `docker build --platform linux/amd64 -t web-server -f Dockerfile.web-server  .`\n- `docker build --platform linux/amd64 -t ui-server -f Dockerfile.ui-server  .`\n- `eval $(minikube docker-env -u)`\n\nInside `dev/minikube-dev` you can run below command for testing all services\n\n- `kubectl apply -f .`\n\nAfter testing\n\n- `kubectl delete -f .`\n\n### Deploy with HELM\n\n- Config maps - used to deploy varitey of services like `prod` and `dev` with helm commands that makes easier configuration and testing -\u003e `redis.cm.yml`, `model-service.cm.yml`, `web-service.cm.yml`.\n- Secrets - used to hold passwords for services like redis -\u003e `redis.secret.yml`\n- Namespace - a tag and environemnt to seperate different deployments and relates services\n- `Volume` - redis has a peresistent volume -\u003e `redis.volume.yml`\n\nCreate a namespace `production` for production deployments\n\n- `kubectl create namespace production`\n\nChange the current environment namespace\n\n- `kubectl config set-context --current --namespace=production`\n\nIn manifest file `value-prod.yaml` i have provided `prod` namespace and `prod` environment so while accessing pods you need\nto query the namespace `prod`\n\n- `helm install fastapi-release-prod fastapi-helm --values fastapi-helm/values.yaml -f fastapi-helm/values-prod.yaml`\n\n![](./assets/snap_helm_install.png)\n\n![](./assets/snap_helm_ls.png)\n\nThese are my services, deployment and pods for prod deployment\n\n![](./assets/snap_get_all_pods.png)\n\nEnsure that in `kubectl get pod -n prod` all pods are in running state\n\nExposes ports to host for testing and usage\n\n- `minikube service model-server-service -n prod`\n\n- `minikube service web-server-service  -n prod`\n\n- `minikube service ui-server-service  -n prod`\n\nOpen the url for `ui-server-service` displayed in CLI\n\n**Only if you want another set of deployments for dev**\n- `kubectl create namespace production`\n- `kubectl config set-context --current --namespace=production`\n- `helm install fastapi-release-dev fastapi-helm --values fastapi-helm/values.yaml -f fastapi-helm/values-dev.yaml`\n\n**Debugging pods or server**\n\n- kubectl logs \u003c`pod name`\u003e\n\n**Testing**\n\n- `curl -X POST \u003chttp://127.0.0.1:43139/classify-catdog\u003e -H \"Content-Type: multipart/form-data\" -F \"image=@dog.jpg\"`\n\n    ```\n    {\"class\": dog, \"confidence\": 0.9892888878}\n    ```\n\nNote: The first prediction can take around 25 seconds as its in a cpu and its a cold start. Successive predictions are faster and i have increased the wait time to 30 seconds in fastapi server. Remember cache while debugging. Also the memory allocated to each pod can also matter during prediciton which can be configured in manifest files\n\n**Ingress Services**\n\nIn `/etc/hosts` add\n\n```\n\n127.0.0.1       fastapi.locahost        localhost\n\n```\n\n- Also do in  `C:\\Windows\\System32\\drivers\\etc\\hosts` if you are in windows and using wsl. if not u can reach through curl in wsl but not in browser.\n\n- `minikube addons enable metrics-server`\n- `minikube addons enable dashboard`\n- `minikube addons enable ingress`\n- `minikube tunnel`\n\n- Note: In screenshot its accessible at `http://fastapi.prod` \n\n![](./assets/snap_ingress_output.png)\n\n![](./assets/snap_ingress_service.png)\n\n- `minikube dashboard` for pod monitoring\n\n### Bonus assignment 1\n\n**Expose to internet with ngrok**\n\nGet auth token from ngrok website and use in below command\n\nAuthentication from ngrok\n\n- `ngrok authtoken \u003cyour ngrok api\u003e`\n\nForward port from pod to local\n\n- `kubectl port-forward service/ui-server-service  8080:80 -n prod`\n\nExpose the port 8080 to internet\n\n- `ngrok http 8080`\n\n    ![](./assets/snap_ngrok_cli.png)\n\n    ![](./assets/snap_ngrok_test_broswer.png)\n\n\n    ![](./assets/snap_different_broswer.png)\n\n### Bonus assignment 2\n\nIn case if you know any front end development with any java script frameworks like React, Angular or SAPUI5 you can refer this video for NextJS development. [Reference Youtube Video](https://www.youtube.com/watch?v=PtDIVU_tlo0)\n\nCreate base next js project with `ui` as the project name\n\n- `npx create-next-app@latest ui`\n\nPress default option for every question in cli\n\n- Just copy the contents of `Components/Simplefileupload.js`, `app/predict/route.js` and `app/page.tsx`.\n\n- Change the port in `package.json` -\u003e `scripts` -\u003e `\"dev\": \"next dev -p 80 --turbopack\"`\n\n- `npm run dev`\n\n- `page.tsx` is the one that shows the main page and `Simplefileupload.js` has the front end code to show upload and prediction output. it calls `predict/route.js` and makes a post call to the specified URL.\n\nNote: if you are developing with some other reference code since its a http request so you may get some error like -\u003e \n- `mixed Content: The page at 'https://refactored-spoon-wq6jwjrq6jf6jg-3000.app.github.dev/' was loaded over HTTPS, but requested an insecure resource` -\u003e Use the `route.ts` to handle it\n\n- Below means the the docker container is not able to access fastapi-web-server service so check with curl and debug\n    ````\n    POST http://localhost:9000/classify-imagenet/ net::ERR_CONNECTION_REFUSED\n    ```\n\n**Testing**\n\n- Tested in two different devices\n\n    ![](./assets/snap_different_broswer.png)\n\n**Uninstall**\n\nUninstalling it brings down all services and pods down\n\n- `helm uninstall fastapi-release-prod`\n\n- if you have started exposing/forwarding any service like this `minikube service model-server-service` press `CNTRL+C` and exit \n\n- `kubectl delete namespace production`\n\n- `minikube delete` -\u003e deletes all containers and images in minikube docker environment\n\n### Learnings\n\n- Learnt a lot about kubernetes and its related services\n- While mounting docker volume make sure the files inside docker also matches\n- Checking status and logs of each pod and debugging\n- Reload a pod `kubectl delete pods -l app=web-server`\n- `Helm charts` and using values for easier configuration which is like `hydra` for `python`\n- using `redis db` and `cache services` with fastapi and serving faster.\n\n- NextJS - `mixed Content: The page at 'https://refactored-spoon-wq6jwjrq6jf6jg-3000.app.github.dev/' was loaded over HTTPS, but requested an insecure resource` -\u003e Use the `route.ts` to handle it.\n\n- NextJS - Below means the the docker container is not able to access fastapi-web-server service so check with curl and debug.\n\n    ````\n    POST http://localhost:9000/classify-imagenet/ net::ERR_CONNECTION_REFUSED\n    ```\n### Results Screenshots\n\n**Reports**\n\nReports of all commands given in assignment\n\n- [reports](./report.md)\n\n**Architecture**\n\n![](./assets/snap_architecture.png)\n\n\n**Helm deployment**\n\n![](./assets/snap_helm_install.png)\n\n![](./assets/snap_helm_ls.png)\n\n![](./assets/snap_get_all_pods.png)\n\n**Deployment testing**\n\n![](./assets/snap_fastapi_docs.png)\n\n![](./assets/snap_base_work.png)\n\n**Ingress test**\n\n![](./assets/snap_ingress_tunnel.png)\n\n![](./assets/snap_ingress_service.png)\n\n![](./assets/snap_ingress_output.png)\n\n\n**Bonus assignment results 1 and 2**\n\n![](./assets/snap_ngrok_cli.png)\n\n![](./assets/snap_ngrok_test_broswer.png)\n\n**Bonus assignment results Internet exposure: Another device**\n\n![](./assets/snap_different_broswer.png)\n\n### Group Members\n\n1. Ajith Kumar V (myself)\n2. Pravin Sagar\n3. Hema M\n\n### References\n\n- In case if you know any front end development with any java script frameworks like React, Angular or SAPUI5 you can refer this video for NextJS development. [Reference Youtube Video](https://www.youtube.com/watch?v=PtDIVU_tlo0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Femlo4-session-14-ajithvcoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajithvcoder%2Femlo4-session-14-ajithvcoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajithvcoder%2Femlo4-session-14-ajithvcoder/lists"}