Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/redhat-na-ssa/demo-triton-yolo
Customize Nvidia Triton to use OpenShift Source to Image building
https://github.com/redhat-na-ssa/demo-triton-yolo
data-science model-serving nvidia openshift triton
Last synced: 4 days ago
JSON representation
Customize Nvidia Triton to use OpenShift Source to Image building
- Host: GitHub
- URL: https://github.com/redhat-na-ssa/demo-triton-yolo
- Owner: redhat-na-ssa
- Created: 2024-10-18T00:57:26.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2024-11-22T22:13:50.000Z (2 months ago)
- Last Synced: 2024-12-04T10:08:51.421Z (2 months ago)
- Topics: data-science, model-serving, nvidia, openshift, triton
- Language: Shell
- Homepage:
- Size: 71.3 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Nvidia Triton for OpenShift Source to Image (s2i)
This repo shows how to package a ML model with a Nvidia Triton server container.
## Quickstart
> [!NOTE]
> In this example Triton server is configured to serve models from:`/models`
### Run Triton - Polling mode (PVC storage)
> [!NOTE]
> You can copy models from local storage via `oc cp`
> to a PVC or ephemeral storage
```sh
oc apply -k gitops/overlays/triton
```
```sh
# change project / ns context
oc project demo-triton
# get pod name
POD=$(oc get pod -l app=triton-server -o custom-columns=POD:.metadata.name --no-headers)
# copy model into /models
oc cp examples/source-builder/models/simple ${POD}:/models/
```
### Run Triton - Use an Init Container to setup model
```sh
oc apply -k gitops/overlays/triton-init
```
### Run Triton - Load models from S3 bucket at runtime
> [!NOTE]
> Modify `triton-s3-models` secret with s3 credentials
```sh
oc apply -k gitops/overlays/triton-s3
```
### Run Builder Demos
```sh
oc apply -k gitops/
```
## More Info
See [examples](examples) for more details
## Links
- [Triton Walkthrough](https://neuralbits.substack.com/p/how-to-use-nvidia-triton-server-the)