{"id":44919149,"url":"https://github.com/jaydu1/fdfi","last_synced_at":"2026-04-29T02:12:18.988Z","repository":{"id":337968682,"uuid":"1120442200","full_name":"jaydu1/FDFI","owner":"jaydu1","description":"Disentangled feature importance","archived":false,"fork":false,"pushed_at":"2026-04-29T00:17:22.000Z","size":1070,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-29T01:24:47.603Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://fdfi.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaydu1.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-21T08:10:56.000Z","updated_at":"2026-04-29T00:16:04.000Z","dependencies_parsed_at":"2026-04-02T12:02:28.055Z","dependency_job_id":null,"html_url":"https://github.com/jaydu1/FDFI","commit_stats":null,"previous_names":["jaydu1/fdfi"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/jaydu1/FDFI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaydu1%2FFDFI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaydu1%2FFDFI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaydu1%2FFDFI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaydu1%2FFDFI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaydu1","download_url":"https://codeload.github.com/jaydu1/FDFI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaydu1%2FFDFI/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32407232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T19:38:08.556Z","status":"online","status_checked_at":"2026-04-29T02:00:06.602Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-18T03:04:00.074Z","updated_at":"2026-04-29T02:12:18.983Z","avatar_url":"https://github.com/jaydu1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FDFI - Flow-Disentangled Feature Importance\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![PyPI](https://img.shields.io/pypi/v/fdfi?label=pypi\u0026color=orange)](https://pypi.org/project/fdfi)\n[![PyPI Downloads](https://static.pepy.tech/personalized-badge/fdfi?period=total\u0026units=INTERNATIONAL_SYSTEM\u0026left_color=BLACK\u0026right_color=BRIGHTGREEN\u0026left_text=downloads)](https://pepy.tech/projects/fdfi)\n\nA Python library for computing feature importance using disentangled methods, inspired by SHAP.\n\nCurrent release: `0.0.5`\n\n## Overview\n\nFDFI (Flow-Disentangled Feature Importance) is a Python module that provides interpretable machine learning explanations through disentangled feature importance methods. This package implements both DFI (Disentangled Feature Importance) and FDFI (Flow-DFI) methods. Similar to SHAP, FDFI helps you understand which features are driving your model's predictions.\n\n## Features\n\n- 🎯 **Multiple Explainer Types**: Tree, Linear, and Kernel explainers for different model types\n- 🧭 **OT-Based DFI**: Gaussian OT (OTExplainer) and Entropic OT (EOTExplainer)\n- 🌊 **Flow-DFI**: FlowExplainer with CPI and SCPI methods for non-Gaussian data\n- 📊 **Rich Visualizations**: Summary, waterfall, force, and dependence plots\n- 🔧 **Easy to Use**: Simple API similar to SHAP\n- 🧪 **Statistical Inference**: Confidence intervals and multiple testing correction (FDR/FWER)\n- 🚀 **Extensible**: Built with modularity in mind for future enhancements\n\n## Installation\n\n### From Source\n\n```bash\ngit clone https://github.com/jaydu1/FDFI.git\ncd FDFI\npip install -e .\n```\n\n### Dependencies\n\nUse `pyproject.toml` extras:\n\n```bash\npip install -e \".[dev]\"\npip install -e \".[plots]\"\npip install -e \".[flow]\"\n```\n\n## Quick Start\n\n```python\nimport numpy as np\nfrom fdfi.explainers import OTExplainer\n\n# Define your model\ndef model(X):\n    return X.sum(axis=1)\n\n# Create background data\nX_background = np.random.randn(100, 10)\n\n# Create an explainer\nexplainer = OTExplainer(model, data=X_background, nsamples=50)\n\n# Explain test instances\nX_test = np.random.randn(10, 10)\nresults = explainer(X_test)\n\n# Confidence intervals (post-hoc)\nci = explainer.conf_int(alpha=0.05, target=\"X\", alternative=\"two-sided\")\n\n# With multiple testing correction (e.g., FDR control)\nci_fdr = explainer.conf_int(multitest_method=\"fdr_bh\")\nexplainer.summary(multitest_method=\"fdr_bh\")\n```\n\n### CI Defaults in v0.0.2\n\nBy default, `conf_int()` now uses:\n\n- `var_floor_method=\"mixture\"`\n- `margin_method=\"mixture\"`\n\nThis improves stability for weak effects and avoids ad hoc thresholding in many use cases.\nYou can still override both methods explicitly if needed.\n\n## EOT Options (Entropic OT)\n\n`EOTExplainer` supports adaptive epsilon, stochastic transport sampling, and\nGaussian/empirical targets:\n\n```python\nfrom fdfi.explainers import EOTExplainer\n\nexplainer = EOTExplainer(\n    model.predict,\n    X_background,\n    auto_epsilon=True,\n    stochastic_transport=True,\n    n_transport_samples=10,\n    target=\"gaussian\",  # or \"empirical\"\n)\nresults = explainer(X_test)\n```\n\n## Flow-DFI with FlowExplainer\n\n`FlowExplainer` uses normalizing flows for non-Gaussian data, supporting both CPI (Conditional Permutation Importance) and SCPI (Sobol-CPI):\n\n- **CPI**: Average predictions first, then squared difference: $(Y - E[f(\\tilde{X})])^2$\n- **SCPI**: Squared differences first, then average: $E[(Y - f(\\tilde{X}_b))^2]$\n\n```python\nfrom fdfi.explainers import FlowExplainer\n\n# Create explainer with CPI (default)\nexplainer = FlowExplainer(\n    model.predict,\n    X_background,\n    fit_flow=True,\n    method='cpi',     # 'cpi', 'scpi', or 'both'\n    num_steps=200,    # flow training steps\n    nsamples=50,      # counterfactual samples\n    sampling_method='resample',  # 'resample', 'permutation', 'normal', 'condperm'\n)\n\nresults = explainer(X_test)\n# results['phi_Z']: Z-space importance\n# results['phi_X']: same as phi_Z (Z-space methods)\n\n# Confidence intervals\nci = explainer.conf_int(alpha=0.05, target=\"Z\", alternative=\"two-sided\")\n```\n\n### Explainer diagnostics (new in v0.0.2)\n\nDisentangled explainers (`OTExplainer`, `EOTExplainer`, and `FlowExplainer`) report two diagnostics with qualitative labels (GOOD / MODERATE / POOR) using consistent `[FDFI][DIAG]` logging:\n\n- **Latent independence (median dCor)** — lower is better (thresholds: \u003c0.10 good, \u003c0.25 moderate).\n- **Distribution fidelity (MMD)** — lower is better (thresholds: \u003c0.05 good, \u003c0.15 moderate).\n\nExample log:\n\n```\n[FDFI][DIAG] Flow Model Diagnostics\n[FDFI][DIAG] Latent independence (median dCor): 0.0421 [GOOD]  → lower is better\n[FDFI][DIAG] Distribution fidelity (MMD):       0.0187 [GOOD]  → lower is better\n```\n\nAccess diagnostics directly:\n\n```python\ndiag = explainer.diagnostics\nprint(diag[\"latent_independence_median\"], diag[\"latent_independence_label\"])\nprint(diag[\"distribution_fidelity_mmd\"], diag[\"distribution_fidelity_label\"])\n```\n\nFor advanced users, flow models can be trained separately:\n\n```python\nfrom fdfi.models import FlowMatchingModel\n\n# Train flow model externally\nflow_model = FlowMatchingModel(X_background, dim=X_background.shape[1])\nflow_model.fit(num_steps=500, verbose='final')\n\n# Set pre-trained flow\nexplainer = FlowExplainer(model.predict, X_background, fit_flow=False)\nexplainer.set_flow(flow_model)\n```\n\n## Project Structure\n\n```\nFDFI/\n├── fdfi/                  # Main package directory\n│   ├── __init__.py       # Package initialization\n│   ├── explainers.py     # Explainer classes\n│   ├── plots.py          # Visualization functions\n│   └── utils.py          # Utility functions\n├── tests/                 # Test suite\n│   ├── test_explainers.py\n│   ├── test_plots.py\n│   └── test_utils.py\n├── docs/                  # Documentation \u0026 tutorials\n│   └── tutorials/        # Jupyter notebook tutorials\n├── pyproject.toml        # Package configuration\n└── README.md            # This file\n```\n\n## Development Status\n\n🚧 **This is starter code for DFI development.** The core structure and API are in place, but full implementations are coming soon.\n\nCurrent status:\n- ✅ Package structure established\n- ✅ Base classes and interfaces defined\n- ✅ Testing framework set up\n- ✅ Documentation structure created\n- 🚧 Core algorithms (in development)\n- 🚧 Visualization functions (in development)\n\n## Testing\n\nRun the test suite:\n\n```bash\n# Install development dependencies\npip install -e \".[dev]\"\n\n# Run tests\npytest\n\n# Run tests with coverage\npytest --cov=fdfi --cov-report=html\n```\n\n## Documentation\n\nFull documentation and tutorials are available in the `docs/` directory:\n- [Quickstart Tutorial](docs/tutorials/quickstart.ipynb)\n- [OT Explainer Tutorial](docs/tutorials/ot_explainer.ipynb)\n- [EOT Explainer Tutorial](docs/tutorials/eot_explainer.ipynb)\n- [Flow Explainer Tutorial](docs/tutorials/flow_explainer.ipynb)\n- [Confidence Intervals](docs/tutorials/confidence_intervals.ipynb)\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## References\n\nFDFI is based on:\n- Du, J.-H., Roeder, K., \u0026 Wasserman, L. (2025). Disentangled Feature Importance. *arXiv preprint arXiv:2507.00260*.\n- Chen, X., Guo, Y., \u0026 Du, J.-H. (2026). Flow-Disentangled Feature Importance. In *The Thirteenth International Conference on Learning Representations (ICLR)*.\n\nRelated work:\n- [SHAP](https://github.com/slundberg/shap): A game theoretic approach to explain machine learning models\n\n## Citation\n\nIf you use DFI in your research, please cite:\n\n```bibtex\n@software{dfi2026,\n  title={DFI: Python Library for Disentangled Feature Importance},\n  author={DFI Team},\n  year={2026},\n  url={https://github.com/jaydu1/FDFI}\n}\n\n@article{du2025disentangled,\n  title={Disentangled Feature Importance},\n  author={Du, Jin-Hong and Roeder, Kathryn and Wasserman, Larry},\n  journal={arXiv preprint arXiv:2507.00260},\n  year={2025}\n}\n\n@inproceedings{chen2026flow,\n  title={Flow-Disentangled Feature Importance},\n  author={Chen, Xin and Guo, Yifan and Du, Jin-Hong},\n  booktitle={The Thirteenth International Conference on Learning Representations},\n  year={2026}\n}\n```\n\n## Contact\n\nFor questions and issues, please use the [GitHub issue tracker](https://github.com/jaydu1/FDFI/issues).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaydu1%2Ffdfi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaydu1%2Ffdfi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaydu1%2Ffdfi/lists"}