{"id":16507865,"url":"https://github.com/noahgift/mlflow-project-best-practices","last_synced_at":"2025-04-09T15:06:16.043Z","repository":{"id":49363424,"uuid":"469228114","full_name":"noahgift/mlflow-project-best-practices","owner":"noahgift","description":"An example MLFlow project","archived":false,"fork":false,"pushed_at":"2025-01-10T22:57:38.000Z","size":2592,"stargazers_count":48,"open_issues_count":1,"forks_count":41,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-09T15:06:11.595Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/noahgift.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-12T23:57:11.000Z","updated_at":"2025-02-05T10:35:32.000Z","dependencies_parsed_at":"2025-02-24T04:11:10.081Z","dependency_job_id":"f2dd4cd7-87b9-4953-8299-751685dd3401","html_url":"https://github.com/noahgift/mlflow-project-best-practices","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noahgift%2Fmlflow-project-best-practices","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noahgift%2Fmlflow-project-best-practices/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noahgift%2Fmlflow-project-best-practices/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noahgift%2Fmlflow-project-best-practices/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/noahgift","download_url":"https://codeload.github.com/noahgift/mlflow-project-best-practices/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055284,"owners_count":21040157,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T15:31:18.444Z","updated_at":"2025-04-09T15:06:16.026Z","avatar_url":"https://github.com/noahgift.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"## 🎓 Pragmatic AI Labs | Join 1M+ ML Engineers\n\n### 🔥 Hot Course Offers:\n* 🤖 [Master GenAI Engineering](https://ds500.paiml.com/learn/course/0bbb5/) - Build Production AI Systems\n* 🦀 [Learn Professional Rust](https://ds500.paiml.com/learn/course/g6u1k/) - Industry-Grade Development\n* 📊 [AWS AI \u0026 Analytics](https://ds500.paiml.com/learn/course/31si1/) - Scale Your ML in Cloud\n* ⚡ [Production GenAI on AWS](https://ds500.paiml.com/learn/course/ehks1/) - Deploy at Enterprise Scale\n* 🛠️ [Rust DevOps Mastery](https://ds500.paiml.com/learn/course/ex8eu/) - Automate Everything\n\n### 🚀 Level Up Your Career:\n* 💼 [Production ML Program](https://paiml.com) - Complete MLOps \u0026 Cloud Mastery\n* 🎯 [Start Learning Now](https://ds500.paiml.com) - Fast-Track Your ML Career\n* 🏢 Trusted by Fortune 500 Teams\n\nLearn end-to-end ML engineering from industry veterans at [PAIML.COM](https://paiml.com)\n\n\n# mlflow-project-best-practices\nAn example MLFlow project\n\n[![Python application test with Github Actions](https://github.com/noahgift/mlflow-project-best-practices/actions/workflows/main.yml/badge.svg)](https://github.com/noahgift/mlflow-project-best-practices/actions/workflows/main.yml)\n\n\n## High Level Architecture\n\n![end-to-end-mlops](https://user-images.githubusercontent.com/58792/158695115-a4f8fd97-fbb5-4f9f-b548-ca424636e0ae.png)\n\nYou can Notebook see the Databricks code here:  https://github.com/noahgift/mlflow-project-best-practices/blob/main/XGBoost-fake-news-automl.ipynb\n\n## Example in Azure CloudShell\n![predict](https://user-images.githubusercontent.com/58792/176550829-79f5be83-48bc-4851-88d7-df59c24fa95f.png)\n\n\n## Example in Github Codespaces\n![Screen Shot 2022-03-12 at 7 11 31 PM](https://user-images.githubusercontent.com/58792/158039417-e6510971-240c-45d2-984f-cd8ab88a1569.png)\n\n### Curl Example\n```\ncurl \\\n  -u token:$DATABRICKS_TOKEN \\\n  -X POST \\\n  -H \"Content-Type: application/json; format=pandas-records\" \\\n  -d@data.json \\\n  https://adb-2951765055089996.16.azuredatabricks.net/model/Fake-News/1/invocations\n```\n\n\n## Deploy Architecture\n\n![IMG_0217](https://user-images.githubusercontent.com/58792/158256886-a0ae8edb-9c69-4ada-b4ca-17a796af6ed7.jpg)\n\n## Query registered model\n\n* [Azure Databricks Model Serving Architecture](https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/model-serving)\n\n## Query with Databricks CLI\n\n* [Databricks CLI](https://docs.databricks.com/dev-tools/cli/index.html)\n\n1. Create a databricks config:\n`touch ~/.databrickscfg`\n2.  Put in host and token\n3.  Query jobs\n\n`databricks jobs list --output JSON | jq`\n4.  List clusters\n\n`databricks clusters list --output JSON | jq `\n\n5.  List contents of DBFS\n\n`databricks fs ls dbfs:/`\n\n## Query for Models with API\n\n* [Refer to Quikstart Python](https://docs.databricks.com/applications/mlflow/quick-start-python.html)\n* [Refer to Create or Register Model](https://docs.microsoft.com/en-us/azure/databricks/applications/machine-learning/manage-model-lifecycle/#create-or-register-a-model)\n* [Access tracking server external](https://docs.databricks.com/applications/mlflow/access-hosted-tracking-server.html)\n\nYou need to set the tracking URI.\n\n```bash\nexport MLFLOW_TRACKING_URI=databricks\n```\n\n```python\nfrom pprint import pprint\nfrom mlflow.tracking import MlflowClient\nclient = MlflowClient()\nfor rm in client.list_registered_models():\n  pprint(dict(rm), indent=4)\n```\n\n\n## Download Model\n\n* [Download model artifacts](https://docs.databricks.com/applications/mlflow/models.html#download-model-artifacts)\n\nCLI version\n\n```\nmlflow artifacts download --artifact-uri models:/\u003cname\u003e/\u003cversion|stage\u003e\n````\n\nTo use Python do the following:\n\n```python\nfrom mlflow.store.artifact.models_artifact_repo import ModelsArtifactRepository\n\nmodel_uri = MlflowClient.get_model_version_download_uri(model_name, model_version)\nModelsArtifactRepository(model_uri).download_artifacts(artifact_path=\"\")\n```\n\n\n## Register model for AWS Sagemaker\n\n* [Register model for AWS Sagemaker](https://docs.databricks.com/applications/mlflow/scikit-learn-model-deployment-on-sagemaker.html)\n\n\n![Screen Shot 2022-03-16 at 1 53 31 PM](https://user-images.githubusercontent.com/58792/158655796-06994cd3-c34f-4cbb-aa2b-34dbcb83d9fc.png)\n\n## Goal for today\n\n* Databricks Tensorflow run\n* Feature Store\n* Model Serving\n\n### Model Serving\nhttps://docs.databricks.com/applications/mlflow/model-serving.html\n\nRun it with `mlserve`\n\n```\n mlflow models serve --model-uri  /workspaces/mlflow-project-best-practices/tf-model\n ```\n \n ## References\n \n * [Watch Youtube Walkthrough](https://studio.youtube.com/video/PUXhWZQW8BI/edit?c=UCNDfiL0D1LUeKWAkRE1xO5Q)\n * [Watch Walkthrough on O'Reilly](https://learning.oreilly.com/videos/mlops-platforms-from/032232022VIDEOPAIML/)\n \n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoahgift%2Fmlflow-project-best-practices","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnoahgift%2Fmlflow-project-best-practices","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoahgift%2Fmlflow-project-best-practices/lists"}