{"id":50557184,"url":"https://github.com/simranshaikh20/xai-agent","last_synced_at":"2026-06-04T08:02:10.741Z","repository":{"id":358588816,"uuid":"1241390629","full_name":"SimranShaikh20/XAI-Agent","owner":"SimranShaikh20","description":"Make Any ML Model Explain Itself","archived":false,"fork":false,"pushed_at":"2026-05-18T03:48:49.000Z","size":791,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-18T05:50:09.651Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SimranShaikh20.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-17T10:29:34.000Z","updated_at":"2026-05-18T03:48:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/SimranShaikh20/XAI-Agent","commit_stats":null,"previous_names":["simranshaikh20/xai-agent"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/SimranShaikh20/XAI-Agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimranShaikh20%2FXAI-Agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimranShaikh20%2FXAI-Agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimranShaikh20%2FXAI-Agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimranShaikh20%2FXAI-Agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SimranShaikh20","download_url":"https://codeload.github.com/SimranShaikh20/XAI-Agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SimranShaikh20%2FXAI-Agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33895175,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-04T08:02:09.169Z","updated_at":"2026-06-04T08:02:10.714Z","avatar_url":"https://github.com/SimranShaikh20.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- SEO \u0026 Discovery Keywords:\nhermes agent, explainable AI, XAI agent, SHAP LIME automation, ML model explainability,\nAI fairness, bias detection, open source AI agent, agentic AI, autonomous ML pipeline,\nstreamlit machine learning app, model interpretability tool, hermes agent challenge,\nfeature importance visualization, plain english ML report, EU AI Act compliance\n--\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n# 🧠 XAI-Agent — Model Whisperer\n\n### *Make any ML model explain itself. In plain English. Automatically.*\n\n[![Hermes Agent](https://img.shields.io/badge/Powered%20by-Hermes%20Agent-6366f1?style=for-the-badge)](https://hermesagent.dev)\n[![Python](https://img.shields.io/badge/Python-3.9%2B-3776ab?style=for-the-badge\u0026logo=python)](https://python.org)\n[![Streamlit](https://img.shields.io/badge/UI-Streamlit-ff4b4b?style=for-the-badge\u0026logo=streamlit)](https://streamlit.io)\n[![DEV Challenge](https://img.shields.io/badge/DEV-Hermes%20Agent%20Challenge-0a0a0a?style=for-the-badge)](https://dev.to/challenges/hermes-agent)\n[![License](https://img.shields.io/badge/License-MIT-22c55e?style=for-the-badge)](LICENSE)\n\n\u003cbr/\u003e\n\n\u003e **The gap nobody talks about:**\n\u003e Data scientists speak SHAP. Managers speak English. Legal speaks risk.\n\u003e XAI-Agent is the autonomous bridge — powered by Hermes Agent's multi-step planning pipeline.\n\n\u003cbr/\u003e\n\n**[🚀 Quick Start](#-quick-start) · [🤖 How It Works](#-how-hermes-agent-powers-this) · [📊 Sample Output](#-real-output-example) · [🎯 Why This Wins](#-why-this-matters)**\n\n\u003c/div\u003e\n\n---\n\n## 🔥 The Problem This Solves\n\nEvery ML team has this meeting:\n\n```\nData Scientist  →  \"The SHAP values show mean concave points has 0.0503 importance\"\nProduct Manager →  \"...great. Should we ship it or not?\"\nLegal Counsel   →  \"Is it biased against any demographic group?\"\nData Scientist  →  \"I'll need 3 days to write that report.\"\nManager         →  \"We needed it yesterday.\"\n```\n\n**This happens everywhere — at startups, banks, hospitals, and governments — every single day.**\n\nExplainable AI (XAI) tools like SHAP and LIME have existed for years. But they require:\n- A data scientist to write custom analysis code for every model\n- ML expertise to interpret the outputs\n- Hours of manual work to turn plots into stakeholder reports\n- A separate fairness audit for bias detection\n- Yet another tool to generate a downloadable document\n\n**There is no single tool that does all of this autonomously. Until now.**\n\n---\n\n## ✅ What XAI-Agent Does\n\nUpload your trained model + dataset. Click one button. Get back:\n\n| Output | What it means |\n|--------|--------------|\n| **Executive Summary** | 3-sentence plain-English overview for your CEO |\n| **SHAP Feature Importance Chart** | Visual ranking of what drives every prediction |\n| **Top 5 Features Explained** | Exactly which inputs matter and why, in human language |\n| **3 Predictions Explained** | Why the model made *that specific decision* for *that specific row* |\n| **Bias \u0026 Fairness Badge** | ✅ green or ⚠️ amber — with evidence, not just a guess |\n| **Downloadable Markdown Report** | Ready to paste into Confluence, Notion, or a PR |\n\n**Total time: under 3 minutes. Zero code required after setup.**\n\n---\n\n## ✨ How XAI-Agent Stands Apart\n\n| Capability | Traditional XAI Workflow | XAI-Agent |\n|-----------|--------------------------|-----------|\n| Setup | Write custom Python per model | Upload 2 files |\n| Explainer selection | Manual — must know SHAP internals | Auto-detects model type, picks best explainer |\n| Output audience | Data scientists only | Everyone — engineers, managers, legal, regulators |\n| Bias detection | Separate tool / manual audit | Built into the pipeline, runs automatically |\n| SHAP version compatibility | Often breaks on 3D output arrays | Handles 2D and 3D SHAP arrays natively |\n| Report format | Raw plots + Jupyter notebooks | Structured plain-English Markdown report |\n| Automation level | Single-shot scripts | Hermes Agent 5-tool autonomous planning loop |\n| Regulatory alignment | Not considered | EU AI Act + GDPR explainability requirements in mind |\n\n---\n\n## 🤖 How Hermes Agent Powers This\n\nXAI-Agent is **not** a wrapper around a single function call. It is a genuine **Hermes Agent autonomous pipeline** — five tools, each with a specific responsibility, chained together so each step's output becomes the next step's input.\n\nThis is exactly the kind of multi-step, multi-tool reasoning that Hermes Agent is built for.\n\n```\n╔══════════════════════════════════════════════════════════════════╗\n║              HERMES AGENT — XAI AUTONOMOUS PIPELINE             ║\n╠═══════╦══════════════════════╦═══════════════════════════════════╣\n║ Step  ║ Tool                 ║ Responsibility                    ║\n╠═══════╬══════════════════════╬═══════════════════════════════════╣\n║  1/5  ║ file_reader          ║ Loads .pkl model + .csv dataset   ║\n║       ║                      ║ Auto-detects: classification vs   ║\n║       ║                      ║ regression. Flags class imbalance ║\n║       ║                      ║ Selects right SHAP explainer type ║\n╠═══════╬══════════════════════╬═══════════════════════════════════╣\n║  2/5  ║ shap_analyzer        ║ Runs SHAP on full dataset sample  ║\n║       ║                      ║ TreeExplainer for tree models     ║\n║       ║                      ║ KernelExplainer as fallback       ║\n║       ║                      ║ Handles 2D + 3D SHAP output arrays║\n║       ║                      ║ Ranks all features by mean |SHAP| ║\n║       ║                      ║ Computes direction: ↑ or ↓        ║\n╠═══════╬══════════════════════╬═══════════════════════════════════╣\n║  3/5  ║ lime_explainer       ║ Selects 3 representative samples  ║\n║       ║                      ║ (one per class for classifiers)   ║\n║       ║                      ║ Runs LIME on each instance        ║\n║       ║                      ║ Converts weights to plain English ║\n╠═══════╬══════════════════════╬═══════════════════════════════════╣\n║  4/5  ║ bias_checker         ║ Scans feature names for sensitive ║\n║       ║                      ║ attributes: age, gender, race,    ║\n║       ║                      ║ income, zip, religion, ethnicity  ║\n║       ║                      ║ Checks prediction class imbalance ║\n║       ║                      ║ Returns evidence-backed verdict   ║\n╠═══════╬══════════════════════╬═══════════════════════════════════╣\n║  5/5  ║ report_writer        ║ Assembles all tool outputs into   ║\n║       ║                      ║ structured Markdown report        ║\n║       ║                      ║ Writes executive summary, feature ║\n║       ║                      ║ table, LIME explanations, bias    ║\n║       ║                      ║ section, and recommendations      ║\n╚═══════╩══════════════════════╩═══════════════════════════════════╝\n```\n\n### Why This Is Genuinely Agentic\n\nMost \"AI agent\" demos call one LLM endpoint and display the result. XAI-Agent does something fundamentally different:\n\n**Context flows between tools.** The model type detected in Step 1 determines which SHAP explainer Step 2 uses. The SHAP feature ranking from Step 2 informs which features Step 3's LIME focuses on. The bias verdict from Step 4 shapes the recommendations in Step 5's report. Every tool is aware of what came before.\n\n**The agent handles failure gracefully.** If TreeExplainer fails, the agent falls back to KernelExplainer automatically — no crash, no error shown to the user. This is what production-grade agentic systems do.\n\n**Real edge cases are handled.** Newer versions of SHAP return 3D arrays `(samples, features, classes)` instead of 2D. The agent detects this and slices `[:, :, 1]` automatically — a bug that breaks every naive SHAP implementation.\n\n---\n\n## 🚀 Quick Start\n\n### Prerequisites\n- Python 3.9+\n- pip\n\n### 1 — Clone\n```bash\ngit clone https://github.com/SimranShaikh20/xai-agent.git\ncd xai-agent\n```\n\n### 2 — Install\n```bash\npip install -r requirements.txt\n```\n\u003e First install takes ~2 minutes (SHAP and LightGBM are large). Subsequent runs are instant.\n\n### 3 — Run\n```bash\nstreamlit run app.py\n```\n\n### 4 — Open browser\n```\nhttp://localhost:8501\n```\n\n### 5 — Test with included files\nUpload `sample_model.pkl` + `sample_dataset.csv` → set target column = `target` → click **Run explainability analysis**.\n\nFull analysis completes in under 3 minutes.\n\n---\n\n## 📊 Real Output Example\n\nRunning XAI-Agent on the included breast cancer dataset (569 patients, 30 tumor measurements, RandomForestClassifier):\n\n### Executive Summary\n\u003e This **RandomForestClassifier** was analyzed across **569 samples** and **30 features**.\n\u003e The single most influential predictor is **'worst area'** — tumors with larger worst-case area measurements are significantly more likely to be predicted as malignant.\n\u003e **No obvious demographic bias was detected** — the dataset contains no sensitive demographic features.\n\n### Top 5 Features by SHAP Importance\n\n| Rank | Feature | SHAP Score | Effect |\n|------|---------|-----------|--------|\n| 1 | worst area | 0.0756 | ↑ increases malignancy prediction |\n| 2 | worst concave points | 0.0538 | ↑ increases malignancy prediction |\n| 3 | mean concave points | 0.0503 | ↑ increases malignancy prediction |\n| 4 | worst perimeter | 0.0489 | ↑ increases malignancy prediction |\n| 5 | worst radius | 0.0401 | ↑ increases malignancy prediction |\n\n### Sample Prediction Explained (Plain English)\n\u003e **Row 0 — Predicted: Class 1 (benign) at 94% confidence**\n\u003e\n\u003e The model is highly confident this tumor is benign. The main reason:\n\u003e **'worst area'** was well below the threshold that typically signals malignancy — this single feature decreased the malignancy prediction by 0.141 points.\n\u003e **'worst concave points'** also supported the benign classification (impact: −0.089).\n\n### Bias \u0026 Fairness\n\u003e ✅ **No demographic bias detected.** The dataset contains no features named age, gender, sex, race, ethnicity, zip code, income, religion, or nationality. Prediction distribution: 62.7% benign, 37.3% malignant — within acceptable range.\n\n---\n\n## 📁 Project Structure\n\n```\nxai-agent/\n│\n├── app.py                    ← Streamlit app + full Hermes Agent pipeline\n│   ├── HermesXAIAgent class  ← Agent with 5 autonomous tools\n│   ├── tool_file_reader()    ← Model + dataset loader\n│   ├── tool_shap_analyzer()  ← Global SHAP explainability\n│   ├── tool_lime_explainer() ← Local LIME explanations\n│   ├── tool_bias_checker()   ← Fairness \u0026 demographic audit\n│   └── tool_report_writer()  ← Plain-English report generator\n│\n├── requirements.txt          ← Pinned Python dependencies\n├── generate_test_data.py     ← Script to regenerate test files\n├── sample_model.pkl          ← Pre-trained RandomForest (breast cancer)\n├── sample_dataset.csv        ← 569 rows × 31 cols (30 features + target)\n├── XAI_Agent_Hermes.ipynb    ← Google Colab version (zero install)\n├── .gitignore\n└── README.md\n```\n\n---\n\n## 🔬 Supported Models \u0026 Auto-Explainer Logic\n\n```python\n# The agent auto-selects the right SHAP explainer:\n\nTREE_MODELS = [\n    \"RandomForestClassifier\",   \"RandomForestRegressor\",\n    \"GradientBoostingClassifier\",\"GradientBoostingRegressor\",\n    \"XGBClassifier\",            \"XGBRegressor\",\n    \"LGBMClassifier\",           \"LGBMRegressor\",\n    \"DecisionTreeClassifier\",   \"ExtraTreesClassifier\"\n]\n\nif model_type in TREE_MODELS:\n    explainer = shap.TreeExplainer(model)   # Fast, exact, recommended\nelse:\n    explainer = shap.KernelExplainer(model) # Universal fallback\n```\n\n**SHAP version compatibility fix (unique to this project):**\n```python\n# New SHAP versions return 3D arrays: (samples, features, classes)\n# This breaks every naive implementation. XAI-Agent handles it:\n\nif isinstance(shap_raw, list):\n    shap_vals = shap_raw[1]              # Old SHAP: list per class\nelif shap_raw.ndim == 3:\n    shap_vals = shap_raw[:, :, 1]       # New SHAP: 3D → slice class 1\nelse:\n    shap_vals = shap_raw                 # Regression: use as-is\n```\n\n---\n\n## 🧰 Tech Stack\n\n| Layer | Technology | Why |\n|-------|-----------|-----|\n| **Agent framework** | Hermes Agent | Multi-step autonomous planning |\n| **Global XAI** | SHAP 0.44+ | Industry standard, model-agnostic |\n| **Local XAI** | LIME 0.2+ | Instance-level explanations |\n| **UI** | Streamlit 1.32+ | Fast, Python-native, no JS needed |\n| **ML frameworks** | scikit-learn, XGBoost, LightGBM | Covers 95% of real-world models |\n| **Visualization** | Matplotlib | Reliable, exportable charts |\n| **Report** | Markdown | Portable — works in GitHub, Notion, Confluence |\n\n---\n\n## 🧪 Use Your Own Model\n\n```python\n# Step 1: Train any model\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.datasets import load_breast_cancer\nimport joblib, pandas as pd\n\ndata = load_breast_cancer()\nX = pd.DataFrame(data.data, columns=data.feature_names)\ny = pd.Series(data.target, name=\"target\")\n\nmodel = RandomForestClassifier(n_estimators=100, random_state=42)\nmodel.fit(X, y)\n\n# Step 2: Save it\njoblib.dump(model, \"my_model.pkl\")\nX.assign(target=y).to_csv(\"my_dataset.csv\", index=False)\n\n# Step 3: Upload both files to XAI-Agent → done\n```\n\nWorks with any model savedvia `joblib.dump()` or `pickle.dump()`.\n\n---\n\n## 💡 Why This Matters Beyond the Challenge\n\n### Regulatory Pressure Is Real and Growing\n\n**EU AI Act (2024):** High-risk AI systems must provide meaningful explanations for automated decisions. Non-compliance: up to €30 million or 6% of global revenue.\n\n**GDPR Article 22:** EU citizens have the right to explanation when subject to automated decision-making. Organizations must be able to explain *why* a model reached a decision.\n\n**US Financial Regulation:** The Equal Credit Opportunity Act requires lenders to explain adverse actions. Banks using ML credit scoring need model explainability built in.\n\n**Healthcare AI:** FDA guidance on AI/ML-based software as a medical device (SaMD) requires transparency about how models make decisions.\n\n### Who Needs This Right Now\n\n- **ML engineers** shipping models to production — need audit trails\n- **Data scientists** presenting to non-technical stakeholders — need plain English\n- **Compliance teams** at banks, hospitals, insurance companies — need bias evidence\n- **Product managers** evaluating ML features — need to understand model behavior\n- **Open-source contributors** — need a free alternative to expensive XAI platforms like Fiddler, Arize, or Arthur AI (which cost $50K+/year)\n\n**XAI-Agent is free, open-source, runs locally, and works in 3 minutes.**\n\n---\n\n## 🔮 What's Next (Roadmap)\n\n- [ ] PDF report export with charts embedded\n- [ ] Support for PyTorch and TensorFlow neural networks (DeepExplainer)\n- [ ] SHAP interaction values for feature pair analysis\n- [ ] Counterfactual explanations (\"what would need to change to flip this prediction?\")\n- [ ] Batch analysis across multiple models for comparison\n- [ ] REST API mode for CI/CD pipeline integration\n- [ ] Streamlit Cloud one-click deployment\n\n---\n\n## 🙏 Acknowledgements\n\n- [Hermes Agent](https://hermesagent.dev) — the autonomous agent framework that makes the pipeline possible\n- [SHAP](https://github.com/slundberg/shap) by Scott Lundberg — the gold standard for ML explainability\n- [LIME](https://github.com/marcotcr/lime) by Marco Tulio Ribeiro — local model-agnostic explanations\n- [Streamlit](https://streamlit.io) — for making Python web apps actually enjoyable to build\n- The [DEV Community](https://dev.to) for running the Hermes Agent Challenge\n\n---\n\n## 👤 Author\n\n**Simran Shaikh** — AI/ML Engineer\n\n- 🐦 DEV: [@SimranShaikh20](https://dev.to/simranshaikh20)\n- 💻 GitHub: [@SimranShaikh20](https://github.com/SimranShaikh20)\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Built for the [DEV Hermes Agent Challenge](https://dev.to/challenges/hermes-agent) — May 2026**\n\n*If this helped you, drop a ⭐ on GitHub and a ❤️ on the DEV post — it means a lot!*\n\n\u003c/div\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimranshaikh20%2Fxai-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimranshaikh20%2Fxai-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimranshaikh20%2Fxai-agent/lists"}