https://github.com/nebuly-ai/nos

Module to Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elastic quotas - Effortless optimization at its finest!
https://github.com/nebuly-ai/nos

gpu kubernetes optimization

Last synced: 6 days ago
JSON representation

Module to Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elastic quotas - Effortless optimization at its finest!

• Host: GitHub
• URL: https://github.com/nebuly-ai/nos
• Owner: nebuly-ai
• License: apache-2.0
• Created: 2022-04-12T13:18:09.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2024-04-21T20:57:30.000Z (9 months ago)
• Last Synced: 2024-05-21T12:44:49.008Z (8 months ago)
• Topics: gpu, kubernetes, optimization
• Language: Go
• Homepage: https://www.nebuly.com/
• Size: 9.78 MB
• Stars: 576
• Watchers: 12
• Forks: 26
• Open Issues: 22
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-tensor-compilers - Nebulgym: Easy-to-use Library to Accelerate AI Training
• awesome-repositories - nebuly-ai/nos - Module to Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elastic quotas - Effortless optimization at its finest! (Go)
• awesome-k8s-resources - nos - `nos` is an open-source platform to efficiently run AI workloads on Kubernetes, increasing GPU utilization and reducing infrastructure and operational costs. (Tools and Libraries / Data Processing and Machine Learning)
• awesome-mlops - Nos - Open-source module for running AI workloads on Kubernetes in an optimized way. (Optimization Tools)
• awesome-production-machine-learning - Nos - ai/nos.svg?style=social) - Nos is an open-source platform to efficiently run AI workloads on Kubernetes, increasing GPU utilization and reducing infrastructure and operational costs. (Training Orchestration)

README

        
# Nebuly Operating System (nos)

![](docs/en/docs/img/nos-logo.png)

---

**Documentation**:  docs.nebuly.com/nos/overview 

If you like the project please support it by leaving a star ✨

---

`nos` is the open-source module to efficiently run AI workloads on Kubernetes,

increasing GPU utilization, cutting down infrastructure costs and improving workloads performance.

Currently, the available features are:

* [Dynamic GPU partitioning](https://nebuly-ai.github.io/nos/dynamic-gpu-partitioning/overview): allow to schedule Pods requesting

fractions of GPU. GPU partitioning is performed automatically in real-time based on the Pods pending and running in

the cluster, so that Pods can request only the resources that are strictly necessary and GPUs are always fully utilized.

* [Elastic Resource Quota management](https://nebuly-ai.github.io/nos/elastic-resource-quota/overview): increase the number of Pods running on the

cluster by allowing namespaces to borrow quotas of reserved resources from other namespaces as long as they are

not using them.

![](docs/en/docs/img/gpu-utilization.png)

## Getting started

### Prerequisites

* Kubernetes v1.23 or newer

* [GPU Support must be enabled](http://nebuly-ai.github.io/nos/prerequisites/#enable-gpu-support)

* [Nebuly k8s-device-plugin](https://github.com/nebuly-ai/k8s-device-plugin) (optional, required only if you want to enable MPS partitioning)

* [cert-manager](https://cert-manager.io/docs/) (optional, but recommended)

### Installation

You can install `nos` using Helm 3 (recommended).

You can find all the available configuration values in the Chart [documentation](https://nebuly-ai.github.io/nos/helm-charts/nos/).

```bash

helm install oci://ghcr.io/nebuly-ai/helm-charts/nos \

  --version 0.1.2 \

  --namespace nebuly-nos \

  --generate-name \

  --create-namespace

```

Alternatively, you can use Kustomize by cloning the repository and running `make deploy`.