https://github.com/inftyai/scheduler-plugins

Scheduler plugins designed for smart scheduling with llmaz.
https://github.com/inftyai/scheduler-plugins

Last synced: about 1 year ago
JSON representation

Scheduler plugins designed for smart scheduling with llmaz.

• Host: GitHub
• URL: https://github.com/inftyai/scheduler-plugins
• Owner: InftyAI
• License: apache-2.0
• Created: 2024-08-27T15:47:41.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2025-02-24T03:02:58.000Z (about 1 year ago)
• Last Synced: 2025-04-12T17:23:42.023Z (about 1 year ago)
• Language: Go
• Homepage:
• Size: 133 KB
• Stars: 5
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

          
# vScheduler

A Kubernetes scheduler designed for smart scheduling with llmaz.

## Plugins

vScheduler maintains multiple plugins for llm workloads scheduling.

### ResourceFungibility Plugin

A `llama2-7B` model can be run on __1xA100__ GPU, can also be run on __1xA10__ GPU, this is what we called fungibility.

With [resourceFungibility](./docs/plugins/resource_fungibility.md) plugin, we can simply achieve this with at most 8 alternative GPU types.