https://github.com/slinusc/deepspeed-mii-container
Launch your own high-performance DeepSpeed-MII server for seamless local LLM deployment. This repository provides a Dockerized solution to serve Hugging Face models (e.g., Mistral-7B) with an OpenAI-compatible API, enabling GPU-accelerated, low-latency inference out of the box.
https://github.com/slinusc/deepspeed-mii-container
container deepspeed docker engine inference llm mii
Last synced: 3 months ago
JSON representation
Launch your own high-performance DeepSpeed-MII server for seamless local LLM deployment. This repository provides a Dockerized solution to serve Hugging Face models (e.g., Mistral-7B) with an OpenAI-compatible API, enabling GPU-accelerated, low-latency inference out of the box.
- Host: GitHub
- URL: https://github.com/slinusc/deepspeed-mii-container
- Owner: slinusc
- License: mit
- Created: 2025-05-31T09:55:42.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-05-31T10:41:36.000Z (4 months ago)
- Last Synced: 2025-05-31T22:48:57.575Z (4 months ago)
- Topics: container, deepspeed, docker, engine, inference, llm, mii
- Homepage: https://hub.docker.com/r/slinusc/deepspeed-mii
- Size: 17.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: LICENSE