https://github.com/etsi-ai/etsi-watchdog
Real-time data drift detection and monitoring for machine learning pipelines.
https://github.com/etsi-ai/etsi-watchdog
drift-detection machine-learning python
Last synced: 9 months ago
JSON representation
Real-time data drift detection and monitoring for machine learning pipelines.
- Host: GitHub
- URL: https://github.com/etsi-ai/etsi-watchdog
- Owner: etsi-ai
- License: bsd-2-clause
- Created: 2025-06-26T11:19:11.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-08-02T14:49:54.000Z (11 months ago)
- Last Synced: 2025-08-02T16:33:14.605Z (11 months ago)
- Topics: drift-detection, machine-learning, python
- Language: Python
- Homepage:
- Size: 79.1 MB
- Stars: 21
- Watchers: 1
- Forks: 26
- Open Issues: 24
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# etsi-watchdog
[](https://pypi.org/project/etsi-watchdog/)
[](https://pepy.tech/projects/etsi-watchdog)
[đź“„ DOCS](https://etsi-ai.github.io/docs/etsi-watchdog.html)
**Real-time data drift detection for machine learning pipelines.**
`etsi-watchdog` is a production-ready Python library for drift detection, version comparison, and real-time monitoring of data streams. Designed for ML practitioners, data scientists, and AI engineers who need reliable data quality insights.
---
## ✨ Features
- PSI-based Drift Detection (more algorithms coming)
- Rolling Monitoring with time-frequency windowing
- Version Drift Comparison between model/data snapshots
- Built-in Visualization & JSON Export
- Minimal Dependencies & Fast Performance
- Clear API, suitable for both research and production
---
## 📦 Installation
```bash
pip install etsi-watchdog
```
---
## ⚡ Quickstart
### 🔹 Drift Detection
```python
from etsi.watchdog import DriftCheck
import pandas as pd
ref = pd.read_csv("reference.csv")
live = pd.read_csv("current.csv")
check = DriftCheck(ref)
results = check.run(live, features=["age", "salary"])
for feat, result in results.items():
print(result.summary())
result.plot()
```
---
### 🔹 Rolling Monitoring
```python
from etsi.watchdog import Monitor
monitor = Monitor(reference_df=ref)
monitor.enable_logging("logs/rolling_log.csv")
results = monitor.watch_rolling(
df=live_data_stream,
window=50,
freq="D",
features=["age", "salary"]
)
```
---
### 🔹 Drift Comparison (A/B)
```python
from etsi.watchdog import DriftComparator
check = DriftCheck(ref)
v1 = check.run(live1, features=["age", "salary"])
v2 = check.run(live2, features=["age", "salary"])
comp = DriftComparator(v1, v2)
print(comp.diff())
```
---
## 🚀 etsi-watchdog v3 — Roadmap & Vision
| Feature Area | Description |
|--------------|-------------|
| **Multi-Algorithm Support** | Support for additional drift metrics:
- Jensen–Shannon Divergence (JSD)
- Wasserstein Distance
- Kolmogorov–Smirnov Test (K-S)
- Tree-based drift (e.g., `DeepDrift`) |
| **Plug-in Architecture** | Drift algorithms will be fully plug-and-play. Custom metric support via `register_drift_function()` API. |
| **Real-Time Stream Hooks** | Support for Kafka/Redis/WebSockets to detect drift on live data streams. |
| **Concept Drift Detection** | Integration with models to detect label or concept drift, not just feature distribution shift. |
| **CLI & YAML Configs** | Full CLI support:
`etsi-watchdog detect --ref ref.csv --live live.csv --features age salary`
+ YAML-based configuration for automated pipelines. |
| **Benchmark Suite** | Built-in benchmarking with synthetic datasets to evaluate metric sensitivity and response time. |
| **Dashboard UI (Optional)** | Lightweight dashboard (Streamlit/FastAPI) for monitoring drift over time visually. |
| **Drift Summary Reports** | Generate PDF/HTML reports with drift summary, top features, histograms, and timestamps. |
| **Sklearn & Pandas Integration** | `DriftCheck` will support `.fit()`/`.transform()` methods like Scikit-learn transformers. |
---
## 🤝 Contributing
Contributions are welcome! 🙌
If you have an idea for a drift metric, integration, or improvement, feel free to open an issue or pull request.
Please refer to [CONTRIBUTING.md](https://github.com/etsi-ai/etsi-watchdog/blob/main/CONTRIBUTING.md) for guidelines.
---
## 🧾 License
Licensed under the **BSD-2-Clause License**.
See the [LICENSE](./LICENSE) file for details.
---
## 👨‍💻 Maintainers
- [@PRIYANSH](https://github.com/PriyanshSrivastava0305)
---