https://github.com/kmkolasinski/triton-server-python-backend-bug

A snippet which demonstrates a potential bug in the triton server
https://github.com/kmkolasinski/triton-server-python-backend-bug

Last synced: 8 days ago
JSON representation

A snippet which demonstrates a potential bug in the triton server

• Host: GitHub
• URL: https://github.com/kmkolasinski/triton-server-python-backend-bug
• Owner: kmkolasinski
• License: mit
• Created: 2024-01-26T19:21:28.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-01-27T07:52:32.000Z (9 months ago)
• Last Synced: 2024-01-27T20:49:56.868Z (9 months ago)
• Language: Python
• Size: 6.84 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# triton-server-python-backend-bug [**SOLVED**]

A snippet which demonstrates a potential bug in the triton server

**SOLVED**: I was using old version of triton server. The issue is resolved in the latest version `nvcr.io/nvidia/tritonserver:23.12-py3`

## Build and Restart server

```bash

docker compose build && docker compose down && docker compose up

```

Note: docker compose down - resolves the issue with error "ln: failed to create symbolic link '/opt/tritonserver/lib/libnvidia-ml.so.1': File exists" after restarting the server

## Run the client (Python 3.11.4)

Setup environment (example using conda)

```bash

conda create --name=triton-bug python==3.11

conda activate triton-bug

pip install -r requirements.txt

```

Open a new terminal and run the client

```bash

python run_client.py

```