https://github.com/sozercan/k8s-distributed-inference

🦄 Distributed Inference on Kubernetes with DRA and MIG
https://github.com/sozercan/k8s-distributed-inference

dra dynamic-resource-allocation inference kubernetes llm multi-instance-gpu nvidia

Last synced: 5 days ago
JSON representation

🦄 Distributed Inference on Kubernetes with DRA and MIG

• Host: GitHub
• URL: https://github.com/sozercan/k8s-distributed-inference
• Owner: sozercan
• License: mit
• Created: 2024-09-16T21:01:47.000Z (about 2 months ago)
• Default Branch: main
• Last Pushed: 2024-09-21T01:44:49.000Z (about 2 months ago)
• Last Synced: 2024-10-16T18:23:43.935Z (20 days ago)
• Topics: dra, dynamic-resource-allocation, inference, kubernetes, llm, multi-instance-gpu, nvidia
• Language: Shell
• Homepage:
• Size: 68.4 KB
• Stars: 0
• Watchers: 1
• Forks: 5
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# k8s-distributed-inference

1. [Set up host](provision-host.md)

2. [Deploy Local AI](deploy-local-ai.md)