{"id":15129942,"url":"https://github.com/microsoft/mlops","last_synced_at":"2025-05-15T03:04:50.848Z","repository":{"id":37588176,"uuid":"184819127","full_name":"microsoft/MLOps","owner":"microsoft","description":"MLOps examples","archived":false,"fork":false,"pushed_at":"2024-08-02T16:21:40.000Z","size":383264,"stargazers_count":1947,"open_issues_count":17,"forks_count":560,"subscribers_count":70,"default_branch":"master","last_synced_at":"2025-05-15T00:08:24.414Z","etag":null,"topics":["azureml","mlops"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-03T20:55:03.000Z","updated_at":"2025-05-14T22:05:42.000Z","dependencies_parsed_at":"2024-01-14T03:48:33.670Z","dependency_job_id":"9b9c0e93-b4e1-46bf-a936-0feda15a5576","html_url":"https://github.com/microsoft/MLOps","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMLOps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMLOps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMLOps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2FMLOps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/MLOps/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264765,"owners_count":22041793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azureml","mlops"],"created_at":"2024-09-26T02:24:53.199Z","updated_at":"2025-05-15T03:04:50.828Z","avatar_url":"https://github.com/microsoft.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"---\npage_type: sample\nlanguages:\n- python\nproducts:\n- azure\n- azure-machine-learning-service\n- azure-devops\ndescription: \"MLOps end to end examples \u0026 solutions. A collection of examples showing different end to end scenarios operationalizing ML workflows with Azure Machine Learning, integrated with GitHub and other Azure services such as Data Factory and DevOps.\"\n---\n\n# Updated MLOps Guidance on Azure (2023)\nTo learn the more about the latest guidance from Microsoft about MLOps review the following links.\n\n- [Azure MLOps (v2) Solution Accelerator](https://github.com/Azure/mlops-v2)\n- [Set up MLOps with Azure DevOps](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-setup-mlops-azureml?view=azureml-api-2\u0026tabs=azure-shell)\n- [Set up MLOps with Github](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-setup-mlops-github-azure-ml?view=azureml-api-2\u0026tabs=azure-shell)\n- [Offical Microsoft Documentatation on MLOps](https://learn.microsoft.com/en-us/azure/machine-learning/concept-model-management-and-deployment?view=azureml-api-2)\n- [Microsoft Learn cournse for intro to MLOps](https://learn.microsoft.com/en-us/training/paths/introduction-machine-learn-operations/)\n- [Microsoft Learn course for E2E MLOps](https://learn.microsoft.com/en-us/training/paths/build-first-machine-operations-workflow/)\n\n\n\n------------------------------\n\n# MLOps on Azure\n- [![Build Status](https://dev.azure.com/aidemos/MLOps/_apis/build/status/microsoft.MLOps?branchName=master)](https://dev.azure.com/aidemos/MLOps/_build/latest?definitionId=96?branchName=master)\n- [Example MLOps Release Pipeline](https://dev.azure.com/customai/DevopsForAI-AML/_release?view=all\u0026_a=releases\u0026definitionId=16)\n- [Official Python Azure MLOps repo](https://github.com/Microsoft/MLOpsPython)\n- [MLOps Architecture Deep Dive video](https://www.youtube.com/watch?v=nst3UAGpiBA)\n\n## What is MLOps?\nMLOps empowers data scientists and app developers to help bring ML models to production. \nMLOps enables you to track / version / audit / certify / re-use every asset in your ML lifecycle and provides orchestration services to streamline managing this lifecycle.\n\n### MLOps podcast\nCheck out the recent TwiML podcast on MLOps [here](https://twimlai.com/twiml-talk-321-enterprise-readiness-mlops-and-lifecycle-management-with-jordan-edwards/) \n\n## How does Azure ML help with MLOps?\nAzure ML contains a number of asset management and orchestration services to help you manage the lifecycle of your model training \u0026 deployment workflows.\n\nWith Azure ML + Azure DevOps you can effectively and cohesively manage your datasets, experiments, models, and ML-infused applications.\n![ML lifecycle](./media/ml-lifecycle.png)\n\n## New MLOps features\n- [Azure DevOps Machine Learning extension](https://marketplace.visualstudio.com/items?itemName=ms-air-aiagility.vss-services-azureml) \n- [Azure ML CLI](https://aka.ms/azmlcli)\n- [Create event driven workflows](https://docs.microsoft.com/azure/machine-learning/service/how-to-use-event-grid) using Azure Machine Learning and Azure Event Grid for scenarios such as triggering retraining pipelines\n- [Set up model training \u0026 deployment with Azure DevOps](https://docs.microsoft.com/en-us/azure/devops/pipelines/targets/azure-machine-learning?view=azure-devops)\n\n\u003e If you are using the Machine Learning DevOps extension, you can access model name and version info using these variables:\n\u003e - Model Name: Release.Artifacts.{alias}.DefinitionName containing model name\n\u003e - Model Version: Release.Artifacts.{alias}.BuildNumber \n\u003e where alias is source alias set while adding the release artifact. \n\n## Getting Started / MLOps Workflow\nAn example repo which exercises our recommended flow can be found [here](https://github.com/Microsoft/MLOpsPython)\n\n## MLOps Best Practices\n### Train Model\n- Data scientists work in topic branches off of master.\n- When code is pushed to the Git repo, trigger a CI (continuous integration) pipeline.\n- First run: Provision infra-as-code (ML workspace, compute targets, datastores).\n- For new code: Every time new code is committed to the repo, run unit tests, data quality checks, train model.\n\nWe recommend the following steps in your CI process:\n- **Train Model** - run training code / algo \u0026 output a [model](https://docs.microsoft.com/en-us/azure/machine-learning/concept-azure-machine-learning-architecture#model) file which is stored in the [run history](https://docs.microsoft.com/en-us/azure/machine-learning/service/concept-azure-machine-learning-architecture#run).\n- **Evaluate Model** - compare the performance of newly trained model with the model in production. If the new model performs better than the production model, the following steps are executed. If not, they will be skipped.\n- **Register Model** - take the best model and register it with the [Azure ML Model registry](https://docs.microsoft.com/en-us/azure/machine-learning/service/concept-azure-machine-learning-architecture#model-registry). This allows us to version control it.\n\n### Operationalize Model\n- You can package and validate your ML model using the **Azure ML CLI**.\n- Once you have registered your ML model, you can use Azure ML + Azure DevOps to deploy it.\n- You can define a **release definition** in Azure Pipelines to help coordinate a release. Using the DevOps extension for Machine Learning, you can include artifacts from Azure ML, Azure Repos, and GitHub as part of your Release Pipeline.\n- In your release definition, you can leverage the Azure ML CLI's **model deploy** command to deploy your Azure ML model to the cloud (ACI or AKS).\n- Define your deployment as a [gated release](https://docs.microsoft.com/en-us/azure/devops/pipelines/release/approvals/gates?view=azure-devops). This means that once the model web service deployment in the Staging/QA environment is successful, a notification is sent to approvers to manually review and approve the release. Once the release is approved, the model scoring web service is deployed to [Azure Kubernetes Service(AKS)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes) and the deployment is tested.\n\n# MLOps Solutions\nWe are committed to providing a collection of best-in-class solutions for MLOps, both in terms of well documented \u0026 fully managed cloud solutions, as well as reusable recipes which can help your organization to bootstrap its MLOps muscle. These examples are community supported and are not guaranteed to be up-to-date as new features enter the product.\n\nAll of our examples will be built in the open and we welcome contributions from the community!\n- https://github.com/Microsoft/MLOpsPython (reference architecture for MLOps + python)\n- https://github.com/Microsoft/Recommenders (recommender systems with E2E mlops baked in)\n- https://github.com/MicrosoftDocs/pipelines-azureml (CI/CD with the azure ML CLI)\n- https://github.com/Microsoft/MLOps_VideoAnomalyDetection (self-supervised learning with hyperparameter tuning and automated retraining)\n- https://github.com/Azure-Samples/MLOpsDatabricks (set up MLOps with Azure ML + databricks)\n- https://github.com/roalexan/azureml#schedule-using-adf  (schedule an azure ML pipeline from an azure data factory pipeline)\n- https://www.azuredevopslabs.com/labs/vstsextend/aml/ (automated template to deploy MLOps on ADO)\n- https://github.com/Azure/ACE_Azure_ML/tree/master/devops (set up azure ML + azure DevOps together for predictive maintenance)\n- https://github.com/microsoft/nlp ( Natural language processing examples using MLOps + GitHub + Azure)\n- https://github.com/microsoft/AIArchitecturesAndPractices\n- https://github.com/danielsc/azureml-debug-training/blob/master/Setting%20up%20VSCode%20Remote%20on%20an%20AzureML%20Notebook%20VM.md - code from a notebook VM in VSCode\n- https://github.com/jomit/SecureAzureMLWorkshop (code + scripts to run workshop around building secure ml platform on azure)\n- https://github.com/Azure/ml-functions-package-demo package an ML model for use in Azure Functions\n- https://github.com/microsoft/seismic-deeplearning (deep learning for seismic imaging and interpretation)\n- https://github.com/CESARDELATORRE/poc-spark-aml/blob/master/spark-job.py (Spark job on aml compute)\n- https://github.com/Azure-Samples/azure-machine-learning-pipeline-observability-sample (Azure Machine Learning Pipeline Run Observability)\n\n## How is MLOps different from DevOps?\n- Data/model versioning != code versioning - how to version data sets as the schema and origin data change\n- Digital audit trail requirements change when dealing with code + (potentially customer) data\n- Model reuse is different than software reuse, as models must be tuned based on input data / scenario.\n- To reuse a model you may need to fine-tune / transfer learn on it (meaning you need the training pipeline)\n- Models tend to decay over time \u0026 you need the ability to retrain them on demand to ensure they remain useful in a production context.\n\n# What are the key challenges we wish to solve with MLOps?\n\n**Model reproducibility \u0026 versioning**\n- Track, snapshot \u0026 manage assets used to create the model\n- Enable collaboration and sharing of ML pipelines\n\n**Model auditability \u0026 explainability**\n- Maintain asset integrity \u0026 persist access control logs\n- Certify model behavior meets regulatory \u0026 adversarial standards\n\n**Model packaging \u0026 validation**\n- Support model portability across a variety of platforms\n- Certify model performance meets functional and latency requirements\n\n**Model deployment \u0026 monitoring**\n- Release models with confidence\n- Monitor \u0026 know when to retrain by analyzing signals such as data drift\n\n# Contributing\n\nThis project welcomes contributions and suggestions.  Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.microsoft.com.\n\nWhen you submit a pull request, a CLA-bot will automatically determine whether you need to provide\na CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions\nprovided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n\n### Related projects\n\n\n[Microsoft AI Labs Github](https://aka.ms/ai-labs) Find other Best Practice projects, and Azure AI design patterns in our central repository. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fmlops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2Fmlops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fmlops/lists"}