{"id":28059562,"url":"https://github.com/amiot99/gpu-inference-microservice","last_synced_at":"2026-04-11T09:45:46.097Z","repository":{"id":291702799,"uuid":"975408226","full_name":"amiot99/gpu-inference-microservice","owner":"amiot99","description":"FastAPI-based microservice for GPU-accelerated image classification using ResNet18 and Docker","archived":false,"fork":false,"pushed_at":"2025-05-08T00:03:39.000Z","size":35,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-12T08:07:02.750Z","etag":null,"topics":["docker","fastapi","gpu","image-classification","inference","machine-learning","microservice","pytorch","resnet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/amiot99.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-30T09:08:39.000Z","updated_at":"2025-05-08T00:03:42.000Z","dependencies_parsed_at":"2025-05-06T05:39:32.386Z","dependency_job_id":null,"html_url":"https://github.com/amiot99/gpu-inference-microservice","commit_stats":null,"previous_names":["amiot99/gpu-inference-microservice"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiot99%2Fgpu-inference-microservice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiot99%2Fgpu-inference-microservice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiot99%2Fgpu-inference-microservice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/amiot99%2Fgpu-inference-microservice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/amiot99","download_url":"https://codeload.github.com/amiot99/gpu-inference-microservice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253700616,"owners_count":21949694,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","fastapi","gpu","image-classification","inference","machine-learning","microservice","pytorch","resnet"],"created_at":"2025-05-12T08:07:07.923Z","updated_at":"2026-04-11T09:45:41.044Z","avatar_url":"https://github.com/amiot99.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"GPU Inference Microservice with FastAPI + PyTorch\n\nThis project is a minimal, production-style microservice for GPU-accelerated AI/ML image classification. \nIt uses pre-trained models (ResNet18 and ResNet50) from PyTorch, wrapped in a FastAPI server, and containerized with Docker.\nAdditionally, it includes Kubernetes deployment configurations for easy scaling and orchestration.\n\nWhat It Does\n\n- Accepts image uploads via a `/predict` endpoint\n- Runs inference using a GPU-enabled ResNet model (user can select either ResNet18 or ResNet50)\n- Returns the predicted class label (e.g., “golden retriever”)\n- Deployable via Docker and Kubernetes\n\nTech Stack \n- PyTorch (ResNet18, pretrained on ImageNet)\n- FastAPI for the web server\n- Docker for containerization\n- Kubernetes for orchestration and scaling\n- Uvicorn for serving the API\n\n\nRun It Yourself:\n\n1. Clone the repo\n   ```bash\n   git clone https://github.com/your-username/gpu-inference-microservice.git\n   cd gpu-inference-microservice\n   ```\n\n2. Build the Docker image - [Docker](https://www.docker.com/)\n   ```\n   docker build -t gpu-inference-app .\n   ```\n3. Run the container\n   ```\n   docker run -p 8000:8000 gpu-inference-app\n   ```\n4. Use the API\n   ```\n   Visit http://localhost:8000/docs\n   Upload an image → get the predicted label\n   ```\n\nRunning on GPU (Optional) \n   ```\n   To enable GPU acceleration with NVIDIA GPUs, you'll need:\n   * NVIDIA GPU drivers\n   * NVIDIA Container Toolkit\n   * Docker with GPU support\n\n   Once set up, use this command to run the container using your GPU:\n\n   docker run --gpus all -p 8000:8000 gpu-inference-app\n   ```\nDeploying with Kubernetes (Optional) \nIf you want to deploy the application using Kubernetes: \n\n1. Ensure Minikube is installed:\n   [Install Minikube](https://minikube.sigs.k8s.io/docs/start/?arch=%2Fwindows%2Fx86-64%2Fstable%2F.exe+download)\n2. Start Minikube:\n   ```\n   minikube start --driver=docker\n   ```\n3. Build the docker image inside Minikube:\n   ```\n   eval $(minikube docker-env)\n   docker build -t gpu-inference-app:latest .\n   ```\n4. Apply the Kubernetes Manifests:\n   ```\n   kubectl apply -f k8s/\n   ```\n5. Verify the Deployments:\n   ```\n   kubectl get pods\n   kubectl get svc\n   ```\n6. Access the Application:\n   ```\n   minikube service gpu-inference-service\n   ```\n   \nTroubleshooting:\n\nIssue: WSL2 disk space not reclaimed after deleting docker containers/images\n\nIf you notice that the disk space in WSL2 is not being reclaimed after deleting Docker containers and images, follow \nthese steps to manually compact the disk:\n\n1. Shut down Docker and WSL2: \n   ```\n   wsl -shutdown\n   ```\n2. Locate the VHDX File:\n   Navigate to the WSL2 virtual disk file. The default path is:\n   ```\n   C:\\Users\\{username}\\AppData\\Local\\Docker\\wsl\\disk\\docker_data.vhdx\n   ```\n3. Compact the Disk:\n   Open PowerShell as Administrator and run:\n   ```\n   diskpart\n   ```\n   In the diskpart prompt, enter the following commands:\n   ```\n   select vdisk file=\"C:\\Users\\{username}\\AppData\\Local\\Docker\\wsl\\disk\\docker_data.vhdx\"\n   attach vdisk readonly\n   compact vdisk\n   detach vdisk\n   exit\n   ```\n4. Restart Docker Desktop\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famiot99%2Fgpu-inference-microservice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famiot99%2Fgpu-inference-microservice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famiot99%2Fgpu-inference-microservice/lists"}