https://github.com/ray-project/mlflow-ray-serve
MLFlow Deployment Plugin for Ray Serve
https://github.com/ray-project/mlflow-ray-serve
Last synced: 9 months ago
JSON representation
MLFlow Deployment Plugin for Ray Serve
- Host: GitHub
- URL: https://github.com/ray-project/mlflow-ray-serve
- Owner: ray-project
- License: apache-2.0
- Created: 2020-12-17T21:11:11.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2022-04-12T17:51:33.000Z (over 4 years ago)
- Last Synced: 2025-04-11T17:46:32.099Z (over 1 year ago)
- Language: Python
- Size: 40 KB
- Stars: 44
- Watchers: 4
- Forks: 10
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MLflow-Ray-Serve
An experimental plugin that integrates [Ray Serve](https://docs.ray.io/en/master/serve/) with the MLflow pipeline.
``mlflow-ray-serve`` enables MLflow users to deploy MLflow models at scale on Ray Serve.
This plugin implements the [Python API](https://www.mlflow.org/docs/latest/python_api/mlflow.deployments.html)
and [command-line interface](https://www.mlflow.org/docs/latest/cli.html#mlflow-deployments) for MLflow deployment plugins.
## Installation
```bash
pip install mlflow-ray-serve
```
The following packages are required and will be installed along with the plugin:
1. `"ray[serve]"`
2. `"mlflow>=1.12.0"`
This plugin requires Ray version 1.7.0 or greater.
## Usage
This plugin must be used with a detached Ray Serve instance running on a Ray cluster. An easy way to set this up is by running the following two commands:
```bash
ray start --head # Start a single-node Ray cluster locally.
serve start # Start a detached Ray Serve instance.
```
The API is summarized below. For full details see the MLflow deployment plugin [Python API](https://www.mlflow.org/docs/latest/python_api/mlflow.deployments.html)
and [command-line interface](https://www.mlflow.org/docs/latest/cli.html#mlflow-deployments) documentation.
See https://github.com/mlflow/mlflow/tree/master/examples/ray_serve for a full example.
### Create deployment
Deploy a model built with MLflow using Ray Serve with the desired [configuration parameters](https://docs.ray.io/en/master/serve/package-ref.html#backend-configuration); for example, `num_replicas`. Currently this plugin only supports the `python_function` flavor of MLflow models, and this is the default flavor.
##### CLI
```bash
mlflow deployments create -t ray-serve -m --name -C num_replicas=
```
##### Python API
```python
from mlflow.deployments import get_deploy_client
target_uri = 'ray-serve'
plugin = get_deploy_client(target_uri)
plugin.create_deployment(
name=,
model_uri=,
config={"num_replicas": 4})
```
### Update deployment
Modify the configuration of a deployed model and/or replace the deployment with a new model URI.
##### CLI
```bash
mlflow deployments update -t ray-serve --name -C num_replicas=
```
##### Python API
```python
plugin.update_deployment(name=, config={"num_replicas": })
```
### Delete deployment
Delete an existing deployment.
##### CLI
```bash
mlflow deployments delete -t ray-serve --name
```
##### Python API
```python
plugin.delete_deployment(name=)
```
### List deployments
List the names of all the models deployed on Ray Serve. Includes models not deployed via this plugin.
##### CLI
```bash
mlflow deployments list -t ray-serve
```
##### Python API
```python
plugin.list_deployments()
```
### Get deployment details
##### CLI
```bash
mlflow deployments get -t ray-serve --name
```
##### Python API
```python
plugin.get_deployment(name=)
```
### Run prediction on deployed model
For the prediction inputs, DataFrame, Tensor and JSON formats are supported by the Python API. To invoke via the command line, pass in the path to a JSON file containing the input.
##### CLI
```bash
mlflow deployments predict -t ray-serve --name --input-path --output-path
```
`output-path` is an optional parameter. Without it, the result will be printed in the terminal.
##### Python API
```python
plugin.predict(name=, df=)
```
### Plugin help
Prints the plugin help string.
##### CLI
```bash
mlflow deployments help -t ray-serve
```