{"id":19399243,"url":"https://github.com/fiddler-labs/fiddler-examples","last_synced_at":"2026-02-26T02:41:43.671Z","repository":{"id":124260523,"uuid":"610450272","full_name":"fiddler-labs/fiddler-examples","owner":"fiddler-labs","description":null,"archived":false,"fork":false,"pushed_at":"2026-02-20T20:39:40.000Z","size":113821,"stargazers_count":18,"open_issues_count":3,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-02-21T00:54:02.138Z","etag":null,"topics":["ai-observability","explainable-ai","model-monitoring"],"latest_commit_sha":null,"homepage":"https://docs.fiddler.ai/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fiddler-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-03-06T19:53:32.000Z","updated_at":"2026-02-20T20:39:44.000Z","dependencies_parsed_at":"2025-12-31T11:02:03.765Z","dependency_job_id":null,"html_url":"https://github.com/fiddler-labs/fiddler-examples","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fiddler-labs/fiddler-examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fiddler-labs%2Ffiddler-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fiddler-labs%2Ffiddler-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fiddler-labs%2Ffiddler-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fiddler-labs%2Ffiddler-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fiddler-labs","download_url":"https://codeload.github.com/fiddler-labs/fiddler-examples/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fiddler-labs%2Ffiddler-examples/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29848634,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T22:37:40.667Z","status":"online","status_checked_at":"2026-02-26T02:00:06.774Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-observability","explainable-ai","model-monitoring"],"created_at":"2024-11-10T11:08:39.859Z","updated_at":"2026-02-26T02:41:43.664Z","avatar_url":"https://github.com/fiddler-labs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"left\"\u003e\n    \u003cimg src=\"quickstart/images/logo.png\"\n         alt=\"Image of Fiddler logo\" width=\"200\"/\u003e\n\u003c/div\u003e\n\n***\n\nThis repo contains example notebooks and accompanying documentation for using Fiddler.\n\n# Examples\n\nThese example notebooks aim to give you a quick start on various Fiddler capabilities using different model tasks, data types, and use cases. They can also serve as a reference guide for setting up the monitoring of your own models in Fiddler.\n\n## Getting Started\n\nUse the projects in this repo to onboard models and data to illustrate ML model and LLM application monitoring, analysis, and protection.\nThis repo contains the example notebooks listed below. You can launch them in a Google Colab environment using the Colab links.\n\n\u003e **Note:** This repository uses Git Large File Storage (Git LFS) for managing large files. \n\u003e Please make sure you have Git LFS installed before cloning this repository. \n\u003e You can find installation instructions at [git-lfs.github.com](https://git-lfs.github.com/).\n\u003e You can find the file types tracked via GitLFS at the `.gitattributes` file (currently only `.csv` files)\n\n## The Basics\n\n* [LLM - Comparison](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_LLM_Comparison.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_LLM_Comparison.ipynb)\n* [LLM - Simple Monitoring Quickstart](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_LLM_Chatbot.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_LLM_Chatbot.ipynb)\n* [ML - Simple Monitoring Quickstart](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Simple_Monitoring.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Simple_Monitoring.ipynb)\n* [Managing Model Versions with Fiddler](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Model_Versions.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Model_Versions.ipynb)\n* [User-defined Feature Impact Upload](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_User_Defined_Feature_Impact.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_User_Defined_Feature_Impact.ipynb)\n\n## Specific Use Cases\n\n* [Image/Computer Vision Model Monitoring ](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Image_Monitoring.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Image_Monitoring.ipynb)\n* [NLP Model Monitoring - Multiclass Classification](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_NLP_Multiclass_Monitoring.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_NLP_Multiclass_Monitoring.ipynb)\n* [Class Imbalance Drift Detection](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Imbalanced_Data.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Imbalanced_Data.ipynb)\n* [Ranking Model - Monitoring](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Ranking_Model.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Ranking_Model.ipynb)\n* [Regression Model - Monitoring](https://github.com/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Regression_Model.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/quickstart/latest/Fiddler_Quickstart_Regression_Model.ipynb)\n\n## Cookbooks\n\nUse-case oriented notebooks that demonstrate end-to-end workflows for solving real AI evaluation and monitoring problems with Fiddler. Each cookbook has a companion guide on the [Fiddler docs site](https://docs.fiddler.ai/developers/cookbooks).\n\n* [RAG Evaluation Fundamentals](https://github.com/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_RAG_Evaluation_Fundamentals.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_RAG_Evaluation_Fundamentals.ipynb) — Evaluate RAG quality with built-in evaluators\n* [RAG Experiments at Scale](https://github.com/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_RAG_Experiments_at_Scale.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_RAG_Experiments_at_Scale.ipynb) — Compare RAG pipeline configurations systematically\n* [Custom Judge Evaluators](https://github.com/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_Custom_Judge_Evaluators.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/fiddler-labs/fiddler-examples/blob/main/cookbooks/Fiddler_Cookbook_Custom_Judge_Evaluators.ipynb) — Create domain-specific evaluation criteria\n\n## Fiddler Utils Package - Admin Automation Library\n\nThe [`fiddler_utils`](./fiddler_utils) package is an admin automation library designed to reduce code duplication across utility scripts and notebooks. While **not part of the official Fiddler SDK**, it is available for both Fiddler field engineers and customers to use and extend.\n\n### Key Capabilities\n\n* **Connection Management** - Multi-instance support for working with multiple Fiddler deployments\n* **FQL Utilities** - Parse, validate, and manipulate Fiddler Query Language expressions\n* **Schema Validation** - Compare and validate model schemas across instances\n* **Asset Management** - Export/import segments, custom metrics, alerts, baselines, charts, and dashboards\n* **Model Operations** - Complete model export/import and comprehensive model comparison\n* **Environment Analysis** - Project and model inventory with statistics and reporting\n\n### Installation\n\nFrom the `fiddler-examples` repository root:\n\n```bash\npip install -e .\n```\n\n### Quick Example\n\n```python\nfrom fiddler_utils import get_or_init, ModelComparator, SegmentManager\n\n# Initialize connection\nget_or_init(url='https://your-instance.fiddler.ai', token='your_token')\n\n# Compare two models\ncomparator = ModelComparator(model_a, model_b)\nresult = comparator.compare_all()\nprint(result.to_markdown())\n\n# Export/import segments\nsegment_mgr = SegmentManager()\nsegments = segment_mgr.export_assets(model_id=source_model.id)\nsegment_mgr.import_assets(target_model_id=target_model.id, assets=segments)\n```\n\n### Documentation\n\nSee the [fiddler_utils README](./fiddler_utils/README.md) for comprehensive documentation, API reference, and usage examples.\n\n### Integration with Utilities\n\nSeveral notebooks in the [`misc-utils`](./misc-utils) directory demonstrate `fiddler_utils` features:\n\n* **[export_import_models.ipynb](./misc-utils/export_import_models.ipynb)** - Complete model export/import workflows\n* **[export_import_model_assets.ipynb](./misc-utils/export_import_model_assets.ipynb)** - Asset-level transfers between models\n* **[compare_models.ipynb](./misc-utils/compare_models.ipynb)** - Comprehensive model comparison\n* **[env_stats.ipynb](./misc-utils/env_stats.ipynb)** - Environment reporting and analysis\n\n## Miscellaneous Utilities\n\nThe [misc-utils](./misc-utils) directory contains utility notebooks for customer success engineers, field AI engineers, and solution engineers. These notebooks provide tools for various administrative tasks and solutions to common challenges when working with Fiddler deployments. See the [misc-utils README](./misc-utils/README.md) for a detailed catalog of available utilities.\n\n## License\n\nThis project is licensed under the MIT license. See the [LICENSE](https://github.com/fiddler-labs/fiddler-examples/blob/main/LICENSE) file for more info.\n\n## Additional Resources\n\n* [Documentation](https://docs.fiddler.ai)\n* [Fiddler Blog](https://www.fiddler.ai/blog)\n* [Fiddler Resource Library](https://www.fiddler.ai/resources)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffiddler-labs%2Ffiddler-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffiddler-labs%2Ffiddler-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffiddler-labs%2Ffiddler-examples/lists"}