{"id":13449170,"url":"https://github.com/kserve/kserve","last_synced_at":"2026-03-14T01:15:38.003Z","repository":{"id":37009327,"uuid":"178075572","full_name":"kserve/kserve","owner":"kserve","description":"Standardized Serverless ML Inference Platform on Kubernetes","archived":false,"fork":false,"pushed_at":"2025-05-05T01:41:21.000Z","size":444048,"stargazers_count":4135,"open_issues_count":490,"forks_count":1169,"subscribers_count":68,"default_branch":"master","last_synced_at":"2025-05-06T17:13:54.861Z","etag":null,"topics":["artificial-intelligence","genai","hacktoberfest","istio","k8s","knative","kserve","kubeflow","kubernetes","llm-inference","machine-learning","mlops","model-interpretability","model-serving","pytorch","service-mesh","sklearn","tensorflow","xgboost"],"latest_commit_sha":null,"homepage":"https://kserve.github.io/website/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kserve.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-03-27T21:14:14.000Z","updated_at":"2025-05-06T17:11:07.000Z","dependencies_parsed_at":"2023-01-17T13:01:17.246Z","dependency_job_id":"e6fcdb30-5adb-481e-85a7-dd879ab6173c","html_url":"https://github.com/kserve/kserve","commit_stats":{"total_commits":1567,"total_committers":271,"mean_commits":5.782287822878229,"dds":0.845564773452457,"last_synced_commit":"e7353d19a2b407a0573f879d67e39b9cd4a3de68"},"previous_names":["kubeflow/kfserving","kserve/kfserving"],"tags_count":49,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kserve%2Fkserve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kserve%2Fkserve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kserve%2Fkserve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kserve%2Fkserve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kserve","download_url":"https://codeload.github.com/kserve/kserve/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252761131,"owners_count":21800124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","genai","hacktoberfest","istio","k8s","knative","kserve","kubeflow","kubernetes","llm-inference","machine-learning","mlops","model-interpretability","model-serving","pytorch","service-mesh","sklearn","tensorflow","xgboost"],"created_at":"2024-07-31T06:00:32.586Z","updated_at":"2026-03-14T01:15:37.997Z","avatar_url":"https://github.com/kserve.png","language":"Python","readme":"# KServe\n[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white)](https://pkg.go.dev/github.com/kserve/kserve)\n[![Coverage Status](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/andyi2it/5174bd748ac63a6e4803afea902e9810/raw/coverage.json)](https://github.com/kserve/kserve/actions/workflows/go.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/kserve/kserve)](https://goreportcard.com/report/github.com/kserve/kserve)\n[![OpenSSF Best Practices](https://bestpractices.coreinfrastructure.org/projects/6643/badge)](https://bestpractices.coreinfrastructure.org/projects/6643)\n[![Releases](https://img.shields.io/github/release-pre/kserve/kserve.svg?sort=semver)](https://github.com/kserve/kserve/releases)\n[![LICENSE](https://img.shields.io/github/license/kserve/kserve.svg)](https://github.com/kserve/kserve/blob/master/LICENSE)\n[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack\u0026style=social)](https://github.com/kserve/community/blob/main/README.md#questions-and-issues)\n[![Gurubase](https://img.shields.io/badge/Gurubase-Ask%20KServe%20Guru-006BFF)](https://gurubase.io/g/kserve)\n\nKServe is a standardized distributed generative and predictive AI inference platform for scalable, multi-framework deployment on Kubernetes.\n\nKServe is being [used by many organizations](https://kserve.github.io/website/docs/community/adopters) and is a [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/) incubating project.\n\nFor more details, visit the [KServe website](https://kserve.github.io/website/).\n\n![KServe](/docs/diagrams/kserve_new.png)\n\n### Why KServe?\n\nSingle platform that unifies Generative and Predictive AI inference on Kubernetes. Simple enough for quick deployments, yet powerful enough to handle enterprise-scale AI workloads with advanced features.\n\n### Features\n\n**Generative AI**\n  * 🧮 **Optimized Backends**: Support for vLLM and llm-d for optimized performance for serving LLMs\n  * 📌 **Standardization**: OpenAI-compatible inference protocol for seamless integration with LLMs\n  * 🚅 **GPU Acceleration**: High-performance serving with GPU support and optimized memory management for large models\n  * 💾 **Model Caching**: Intelligent model caching to reduce loading times and improve response latency for frequently used models\n  * 🗂️ **KV Cache Offloading**: Advanced memory management with KV cache offloading to CPU/disk for handling longer sequences efficiently\n  * 📈 **Autoscaling**: Request-based autoscaling capabilities optimized for generative workload patterns\n  * 🔧 **Hugging Face Ready**: Native support for Hugging Face models with streamlined deployment workflows\n\n**Predictive AI**\n  * 🧮 **Multi-Framework**: Support for TensorFlow, PyTorch, scikit-learn, XGBoost, ONNX, and more\n  * 🔀 **Intelligent Routing**: Seamless request routing between predictor, transformer, and explainer components with automatic traffic management\n  * 🔄 **Advanced Deployments**: Canary rollouts, inference pipelines, and ensembles with InferenceGraph\n  * ⚡ **Autoscaling**: Request-based autoscaling with scale-to-zero for predictive workloads\n  * 🔍 **Model Explainability**: Built-in support for model explanations and feature attribution to understand prediction reasoning\n  * 📊 **Advanced Monitoring**: Enables payload logging, outlier detection, adversarial detection, and drift detection\n  * 💰 **Cost Efficient**: Scale-to-zero on expensive resources when not in use, reducing infrastructure costs\n\n### Learn More\nTo learn more about KServe, how to use various supported features, and how to participate in the KServe community, \nplease follow the [KServe website documentation](https://kserve.github.io/website). \nAdditionally, we have compiled a list of [presentations and demos](https://kserve.github.io/website/docs/community/presentations) to dive through various details.\n\n### :hammer_and_wrench: Installation\n\n#### Standalone Installation\n- **[Standard Kubernetes Installation](https://kserve.github.io/website/docs/admin-guide/overview#raw-kubernetes-deployment)**: Compared to Serverless Installation, this is a more **lightweight** installation. However, this option does not support canary deployment and request based autoscaling with scale-to-zero.\n- **[Knative Installation](https://kserve.github.io/website/docs/admin-guide/overview#serverless-deployment)**: KServe by default installs Knative for **serverless deployment** for InferenceService.\n- **[ModelMesh Installation](https://kserve.github.io/website/docs/admin-guide/overview#modelmesh-deployment)**: You can optionally install ModelMesh to enable **high-scale**, **high-density** and **frequently-changing model serving** use cases. \n- **[Quick Installation](https://kserve.github.io/website/docs/getting-started/quickstart-guide)**: Install KServe on your local machine.\n\n#### Kubeflow Installation\nKServe is an important addon component of Kubeflow, please learn more from the [Kubeflow KServe documentation](https://www.kubeflow.org/docs/external-add-ons/kserve/kserve). Check out the following guides for running [on AWS](https://awslabs.github.io/kubeflow-manifests/main/docs/component-guides/kserve) or [on OpenShift Container Platform](https://github.com/kserve/kserve/blob/master/docs/OPENSHIFT_GUIDE.md).\n\n### :flight_departure: [Create your first InferenceService](https://kserve.github.io/website/docs/getting-started/genai-first-isvc)\n\n### :bulb: [Roadmap](./ROADMAP.md)\n\n### :blue_book: [InferenceService API Reference](https://kserve.github.io/website/docs/reference/crd-api)\n\n### :toolbox: [Developer Guide](https://kserve.github.io/website/docs/developer-guide)\n\n### :writing_hand: [Contributor Guide](https://kserve.github.io/website/docs/developer-guide/contribution)\n\n### :handshake: [Adopters](https://kserve.github.io/website/docs/community/adopters)\n\n### Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=kserve/kserve\u0026type=Date)](https://www.star-history.com/#kserve/kserve\u0026Date)\n\n### Contributors\n\nThanks to all of our amazing contributors!\n\n\u003ca href=\"https://github.com/kserve/kserve/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=kserve/kserve\" /\u003e\n\u003c/a\u003e\n","funding_links":[],"categories":["Jsonnet","Python","🎯 Tool Categories","Large Scale Deployment","Go","ML Platforms","Ecosystem Projects","Uncategorized","其他_机器学习与深度学习","Deployment and Serving","Model Deployment","artificial-intelligence","AI \u0026 Machine Learning Platforms","🚀 MLOps","📋 Contents","Inference"],"sub_categories":["🏆 Top Serving Platforms","ML Platforms","Uncategorized","Tools","📊 8. MLOps / LLMOps \u0026 Production","Inference Platform"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkserve%2Fkserve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkserve%2Fkserve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkserve%2Fkserve/lists"}