https://github.com/kserve/modelmesh

Distributed Model Serving Framework
https://github.com/kserve/modelmesh

Last synced: 11 months ago
JSON representation

Distributed Model Serving Framework

• Host: GitHub
• URL: https://github.com/kserve/modelmesh
• Owner: kserve
• License: apache-2.0
• Created: 2021-07-09T23:09:55.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2025-03-13T13:47:45.000Z (about 1 year ago)
• Last Synced: 2025-04-04T00:25:03.476Z (12 months ago)
• Language: Java
• Size: 776 KB
• Stars: 159
• Watchers: 21
• Forks: 74
• Open Issues: 22
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-java - ModelMesh

README

          
[![Build](https://github.com/kserve/modelmesh/actions/workflows/build.yml/badge.svg?branch=main)](https://github.com/kserve/modelmesh/actions/workflows/build.yml)

# ModelMesh

The ModelMesh framework is a mature, general-purpose model serving management/routing layer designed for high-scale, high-density and frequently-changing model use cases. It works with existing or custom-built model servers and acts as a distributed LRU cache for serving runtime models.

For full Kubernetes-based deployment and management of ModelMesh clusters and models, see the [ModelMesh Serving](https://github.com/kserve/modelmesh-serving) repo. This includes a separate controller and provides K8s custom resource based management of ServingRuntimes and InferenceServices along with common, abstracted handling of model repository storage and ready-to-use integrations with some existing OSS model servers.

For more information on supported features and design details, see [these charts](https://github.com/kserve/modelmesh/files/8854091/modelmesh-jun2022.pdf).

## Get Started

To learn more about and get started with the ModelMesh framework, check out [the documentation](/docs).

## Developer guide

Use the [developer guide](developer-guide.md) to learn about development practices for the project.