{"id":20446346,"url":"https://github.com/ray-project/mlflow-ray-serve","last_synced_at":"2025-10-26T08:43:51.153Z","repository":{"id":39484865,"uuid":"322413985","full_name":"ray-project/mlflow-ray-serve","owner":"ray-project","description":"MLFlow Deployment Plugin for Ray Serve","archived":false,"fork":false,"pushed_at":"2022-04-12T17:51:33.000Z","size":41,"stargazers_count":44,"open_issues_count":6,"forks_count":10,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-11T17:46:32.099Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ray-project.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-17T21:11:11.000Z","updated_at":"2025-02-27T08:49:00.000Z","dependencies_parsed_at":"2022-08-09T14:49:25.385Z","dependency_job_id":null,"html_url":"https://github.com/ray-project/mlflow-ray-serve","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Fmlflow-ray-serve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Fmlflow-ray-serve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Fmlflow-ray-serve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ray-project%2Fmlflow-ray-serve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ray-project","download_url":"https://codeload.github.com/ray-project/mlflow-ray-serve/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650420,"owners_count":21139672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T10:19:32.706Z","updated_at":"2025-10-26T08:43:51.059Z","avatar_url":"https://github.com/ray-project.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MLflow-Ray-Serve\n\nAn experimental plugin that integrates [Ray Serve](https://docs.ray.io/en/master/serve/) with the MLflow pipeline.\n``mlflow-ray-serve`` enables MLflow users to deploy MLflow models at scale on Ray Serve.\n\nThis plugin implements the [Python API](https://www.mlflow.org/docs/latest/python_api/mlflow.deployments.html)\nand [command-line interface](https://www.mlflow.org/docs/latest/cli.html#mlflow-deployments) for MLflow deployment plugins.\n\n## Installation\n\n```bash\npip install mlflow-ray-serve\n```\n\nThe following packages are required and will be installed along with the plugin:\n\n1. `\"ray[serve]\"`\n2. `\"mlflow\u003e=1.12.0\"`\n\nThis plugin requires Ray version 1.7.0 or greater.\n\n## Usage\nThis plugin must be used with a detached Ray Serve instance running on a Ray cluster.  An easy way to set this up is by running the following two commands:\n\n```bash\nray start --head # Start a single-node Ray cluster locally.\nserve start # Start a detached Ray Serve instance.\n```\n\nThe API is summarized below. For full details see the MLflow deployment plugin [Python API](https://www.mlflow.org/docs/latest/python_api/mlflow.deployments.html)\nand [command-line interface](https://www.mlflow.org/docs/latest/cli.html#mlflow-deployments) documentation.\n\nSee https://github.com/mlflow/mlflow/tree/master/examples/ray_serve for a full example.\n\n### Create deployment\nDeploy a model built with MLflow using Ray Serve with the desired [configuration parameters](https://docs.ray.io/en/master/serve/package-ref.html#backend-configuration); for example, `num_replicas`.  Currently this plugin only supports the `python_function` flavor of MLflow models, and this is the default flavor.\n\n##### CLI\n```bash\nmlflow deployments create -t ray-serve -m \u003cmodel uri\u003e --name \u003cdeployment name\u003e -C num_replicas=\u003cnumber of replicas\u003e\n```\n\n##### Python API\n```python\nfrom mlflow.deployments import get_deploy_client\ntarget_uri = 'ray-serve'\nplugin = get_deploy_client(target_uri)\nplugin.create_deployment(\n    name=\u003cdeployment name\u003e,\n    model_uri=\u003cmodel uri\u003e,\n    config={\"num_replicas\": 4})\n```\n\n### Update deployment\nModify the configuration of a deployed model and/or replace the deployment with a new model URI.\n\n##### CLI\n```bash\nmlflow deployments update -t ray-serve --name \u003cdeployment name\u003e -C num_replicas=\u003cnew number of replicas\u003e\n```\n\n##### Python API\n```python\nplugin.update_deployment(name=\u003cdeployment name\u003e, config={\"num_replicas\": \u003cnew number of replicas\u003e})\n```\n\n### Delete deployment\nDelete an existing deployment.\n\n##### CLI\n```bash\nmlflow deployments delete -t ray-serve --name \u003cdeployment name\u003e\n```\n\n##### Python API\n```python\nplugin.delete_deployment(name=\u003cdeployment name\u003e)\n```\n\n### List deployments\nList the names of all the models deployed on Ray Serve.  Includes models not deployed via this plugin.\n\n##### CLI\n```bash\nmlflow deployments list -t ray-serve\n```\n\n##### Python API\n```python\nplugin.list_deployments()\n```\n\n### Get deployment details\n\n##### CLI\n```bash\nmlflow deployments get -t ray-serve --name \u003cdeployment name\u003e\n```\n\n##### Python API\n```python\nplugin.get_deployment(name=\u003cdeployment name\u003e)\n```\n\n### Run prediction on deployed model\nFor the prediction inputs, DataFrame, Tensor and JSON formats are supported by the Python API.  To invoke via the command line, pass in the path to a JSON file containing the input.\n\n##### CLI\n```bash\nmlflow deployments predict -t ray-serve --name \u003cdeployment name\u003e --input-path \u003cinput file path\u003e --output-path \u003coutput file path\u003e\n```\n\n`output-path` is an optional parameter. Without it, the result will be printed in the terminal.\n\n##### Python API\n```python\nplugin.predict(name=\u003cdeployment name\u003e, df=\u003cprediction input\u003e)\n```\n\n### Plugin help\nPrints the plugin help string.\n\n##### CLI\n```bash\nmlflow deployments help -t ray-serve\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fray-project%2Fmlflow-ray-serve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fray-project%2Fmlflow-ray-serve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fray-project%2Fmlflow-ray-serve/lists"}