https://github.com/kamalkraj/stable-diffusion-tritonserver
Deploy stable diffusion model with onnx/tenorrt + tritonserver
https://github.com/kamalkraj/stable-diffusion-tritonserver
deploy docker fp16 inference machine-learning nvidia onnx python3 pytorch stablediffusion tensorrt tensorrt-inference transformers triton-inference-server
Last synced: 29 days ago
JSON representation
Deploy stable diffusion model with onnx/tenorrt + tritonserver
- Host: GitHub
- URL: https://github.com/kamalkraj/stable-diffusion-tritonserver
- Owner: kamalkraj
- License: apache-2.0
- Created: 2022-08-31T19:13:43.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2023-08-15T10:21:07.000Z (over 1 year ago)
- Last Synced: 2025-03-25T22:21:21.041Z (about 2 months ago)
- Topics: deploy, docker, fp16, inference, machine-learning, nvidia, onnx, python3, pytorch, stablediffusion, tensorrt, tensorrt-inference, transformers, triton-inference-server
- Language: Jupyter Notebook
- Homepage:
- Size: 2.62 MB
- Stars: 123
- Watchers: 4
- Forks: 18
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# stable-diffusion-tritonserver
Please checkout branch [v2](https://github.com/kamalkraj/stable-diffusion-tritonserver/tree/v2) for converting new models
Please checkout branch [v3](https://github.com/kamalkraj/stable-diffusion-tritonserver/tree/v2) for converting models to TensorRT for fastest inference
## Download models
```bash
# clone this repo
git clone https://github.com/kamalkraj/stable-diffusion-tritonserver.git
cd stable-diffusion-tritonserver
# clone model repo from huggingface
git lfs install
git clone https://huggingface.co/kamalkraj/stable-diffusion-v1-4-onnx
```
Unzip the model weights
```bash
cd stable-diffusion-v1-4-onnx
tar -xvzf models.tar.gz
```
## Triton Inference Server
### Build
```bash
docker build -t tritonserver .
```
### Run
```
docker run -it --rm --gpus all -p8000:8000 -p8001:8001 -p8002:8002 --shm-size 16384m \
-v $PWD/stable-diffusion-v1-4-onnx/models:/models tritonserver \
tritonserver --model-repository /models/
```
## Inference
Install `tritonclient` and run the [notebook](Inference.ipynb) for inference.
```bash
pip install "tritonclient[http]"
```
## Credits
- ONNX conversion script from - [harishanand95/diffusers](https://github.com/harishanand95/diffusers/blob/dml/examples/inference/save_onnx.py)