https://github.com/fastmachinelearning/supersonic
Server infrastructure for GPU inference-as-a-service in large scientific experiments
https://github.com/fastmachinelearning/supersonic
Last synced: 8 months ago
JSON representation
Server infrastructure for GPU inference-as-a-service in large scientific experiments
- Host: GitHub
- URL: https://github.com/fastmachinelearning/supersonic
- Owner: fastmachinelearning
- License: apache-2.0
- Created: 2024-10-22T14:22:41.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-03-14T16:30:50.000Z (9 months ago)
- Last Synced: 2025-04-12T22:18:18.181Z (8 months ago)
- Language: JSON
- Homepage: http://fastmachinelearning.org/SuperSONIC/
- Size: 8.83 MB
- Stars: 5
- Watchers: 15
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
[](https://doi.org/10.5281/zenodo.14815348)
[](https://artifacthub.io/packages/search?repo=supersonic)
SuperSONIC
The [SuperSONIC](http://fastmachinelearning.org/SuperSONIC/ "SuperSONIC") project implements server infrastructure for **inference-as-a-service**
applications in large high energy physics (HEP) and multi-messenger astrophysics
(MMA) experiments. The server infrastructure is designed for deployment at [Kubernetes](https://kubernetes.io) clusters equipped with GPUs.
The main components of SuperSONIC are:
- [Nvidia Triton](https://developer.nvidia.com/triton-inference-server) inference servers
- Dynamic muti-purpose [Envoy Proxy](envoyproxy.io):
- Load balancing
- Rate limiting
- GPU saturation prevention
- Token-based authentication
- (optional) Load-based autoscaling via [KEDA](keda.sh)
- (optional) [Prometheus](https://prometheus.io) instance (deploy custom or connect to existing)
- (optional) Pre-configured [Grafana](https://grafana.com) dashboard
- (optional) [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) and [Grafana Tempo](https://grafana.com/docs/tempo/latest/) for advanced monitoring.
## Installation
The installation is done via a custom Helm plugin which takes care of
internal connectivity of the chart components. Standard Helm installation
is also supported, but requires a lot more manual configuration.
```
helm plugin install https://github.com/fastmachinelearning/SuperSONIC/
helm install-supersonic -n -f
```
Installer plugin usage:
```
Usage:
helm install-supersonic [RELEASE_NAME] [flags]
Flags:
-h, --help Show this help message
-f, --values Specify values file for custom configuration
-n, --namespace Specify Kubernetes namespace for deployment
--version Specify chart version (default: latest version)
Note: Ignored if --local flag is set
--local Install from local chart path instead of remote repository
--path Local chart path (default: ./helm/supersonic)
Only used when --local flag is set
Additional flags will be passed directly to the 'helm install' command
```
To construct the `values.yaml` file for your application, follow [Configuration guide](http://fastmachinelearning.org/SuperSONIC/configuration-guide.html "Configuration guide").
The full list of configuration parameters is available in the [Configuration reference](http://fastmachinelearning.org/SuperSONIC/configuration-reference.html "Configuration reference").
## Server diagram
## Grafana dashboard
## Status of deployment
| | **[CMS](https://home.cern/science/experiments/cms)** | **[ATLAS](https://home.cern/science/experiments/atlas)** | **[IceCube](https://icecube.wisc.edu)** |
|:---|:---:|:---:|:---:|
| **[Purdue Geddes](https://www.rcac.purdue.edu/compute/geddes)** | ✅ | - | - |
| **[Purdue Anvil](https://www.rcac.purdue.edu/compute/anvil)** | ✅ | - | - |
| **[NRP Nautilus](https://docs.nationalresearchplatform.org)** | ✅ | ✅ | ✅ |
| **[UChicago](https://af.uchicago.edu/)** | - | ✅ | - |