{"id":39698116,"url":"https://github.com/databricks-solutions/mlflow-demo","last_synced_at":"2026-01-18T10:18:57.436Z","repository":{"id":308631380,"uuid":"1032772389","full_name":"databricks-solutions/mlflow-demo","owner":"databricks-solutions","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-28T00:04:50.000Z","size":1134,"stargazers_count":2,"open_issues_count":0,"forks_count":3,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-28T07:58:05.201Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/databricks-solutions.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS.txt","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-05T19:58:23.000Z","updated_at":"2025-08-28T00:04:54.000Z","dependencies_parsed_at":"2025-08-28T02:04:49.269Z","dependency_job_id":"eb514b8c-ab17-440e-9bf0-7d32e05c8627","html_url":"https://github.com/databricks-solutions/mlflow-demo","commit_stats":null,"previous_names":["databricks-solutions/mlflow-demo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/databricks-solutions/mlflow-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks-solutions%2Fmlflow-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks-solutions%2Fmlflow-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks-solutions%2Fmlflow-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks-solutions%2Fmlflow-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/databricks-solutions","download_url":"https://codeload.github.com/databricks-solutions/mlflow-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databricks-solutions%2Fmlflow-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28534316,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T10:13:46.436Z","status":"ssl_error","status_checked_at":"2026-01-18T10:13:11.045Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-18T10:18:55.244Z","updated_at":"2026-01-18T10:18:57.412Z","avatar_url":"https://github.com/databricks-solutions.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MLflow 3 GenAI Demo\n\nA comprehensive demonstration of **MLflow 3's GenAI capabilities** for observability and evaluating, monitoring, and improving GenAI application quality. This interactive demo showcases a sales email generation use case with end-to-end quality assessment workflows.\n\nThis interactive demo is deployed as a Databricks app in your Databricks workspace. There is a guided UI experience that's accompanied by Notebooks that show you how to do the end-to-end workflow of evaluating quality, iterating to improve quality, and monitoring quality in production.\n\n**Learn more about MLflow 3:**\n\n- Read the [blog post](https://www.databricks.com/blog/mlflow-30-unified-ai-experimentation-observability-and-governance)\n- View our [website](https://www.managed-mlflow.com/genai)\n- Get started via the [documentation](https://docs.databricks.com/aws/en/mlflow3/genai/)\n\n\u003cvideo src=\"https://i.imgur.com/MXhaayF.mp4\" controls width=\"100%\"\u003e\u003c/video\u003e\n\n## Installing the demo\n\nChoose your installation method:\n\n### 🤖 Option A: Automated Setup (Recommended)\n\n**Estimated time: 2 minutes user input + 15 minutes waiting for scripts to run**\n\nThe automated setup handles resource creation, configuration, and deployment for you using the Databricks Workspace SDK.\n\n#### Prerequisites\n- [ ] **Databricks workspace access** - [Create one here](https://signup.databricks.com/?destination_url=/ml/experiments-signup?source=TRY_MLFLOW\u0026dbx_source=TRY_MLFLOW\u0026signup_experience_step=EXPRESS\u0026provider=MLFLOW\u0026utm_source=email_demo_github) if needed\n- [ ] **Install Python `\u003e=3.10.16`**\n\n#### Run Automated Setup\n\nThe `./auto-setup.sh` script will run all the steps outlined in the [Manual Setup](#-option-b-manual-setup) workflow.\n\n\n- [ ] **1. Install the Databricks CLI \u003e= 0.262.0**\n  - Follow the [installation guide](https://docs.databricks.com/aws/en/dev-tools/cli/install)\n  - Verify installation: Run `databricks --version` to confirm it's installed\n- [ ] **2. Install Python \u003e= 3.10.16**\n- [ ] **3. Authenticate with your workspace**\n  - Run `databricks auth login` and follow the prompts\n  - Configure a profile named `DEFAULT`\n- [ ] **3. Clone repo and run setup script**\n\n    ```bash\n    git clone https://github.com/databricks-solutions/mlflow-demo.git\n    cd mlflow-demo\n    ./auto-setup.sh\n    ```\n\n\n### 🔧 Option B: Manual Setup\n\n**Estimated time: 10 minutes work + 15 minutes waiting for scripts to run**\n\nFor step-by-step manual installation instructions, see **[MANUAL_SETUP.md](MANUAL_SETUP.md)**.\n\nThe manual setup includes:\n- Phase 1: Prerequisites setup (workspace, app creation, MLflow experiment, etc.)\n- Phase 2: Local installation and testing\n- Phase 3: Deployment and permission configuration\n\n---\n\n## MLflow 3 overview\n\nMLflow 3.0 has been redesigned for the GenAI era. If your team is building GenAI-powered apps, this update makes it dramatically easier to evaluate, monitor, and improve them in production.\n\n### Key capabilities\n\n- **🔍 GenAI Observability at Scale:** Monitor \u0026 debug GenAI apps anywhere \\- deployed on Databricks or ANY cloud \\- with production-scale real-time tracing and enhanced UIs. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/tracing/)\n- 📊 **Revamped GenAI Evaluation:** Evaluate app quality using a brand-new SDK, simpler evaluation interface and a refreshed UI. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/)\n- ⚙️ **Customizable Evaluation:** Tailor AI judges or custom metrics to your use case. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/custom-judge/)\n- 👀 **Monitoring:** Schedule automatic quality evaluations (beta). [Link](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/run-scorer-in-prod)\n- 🧪 **Leverage Production Logs to Improve Quality:** Turn real user traces into curated, versioned evaluation datasets to continuously improve app performance . [Link](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/build-eval-dataset)\n- 📝 **Close the Loop with** **Feedback:** Capture end-user feedback from your app’s UI. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/tracing/collect-user-feedback/)\n- **👥 Domain Expert Labeling:** Send traces to human experts for ground truth or target output labeling. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/expert-feedback/label-existing-traces)\n- 📁 **Prompt Management:** Prompt Registry for versioning. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/prompt-version-mgmt/prompt-registry/create-and-edit-prompts)\n- 🧩 **App Version Tracking:** Link app versions to quality evaluations. [Link](https://docs.databricks.com/aws/en/mlflow3/genai/prompt-version-mgmt/version-tracking/track-application-versions-with-mlflow)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabricks-solutions%2Fmlflow-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabricks-solutions%2Fmlflow-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabricks-solutions%2Fmlflow-demo/lists"}