{"id":45253120,"url":"https://github.com/emredeveloper/explainable-agent-lab","last_synced_at":"2026-05-03T17:04:55.670Z","repository":{"id":339060339,"uuid":"1160313933","full_name":"emredeveloper/explainable-agent-lab","owner":"emredeveloper","description":"A local-first, explainable AI agent framework. Features self-healing, detailed error diagnostics, interactive tool-calling traces, and custom dataset evaluation for Ollama \u0026 LM Studio.","archived":false,"fork":false,"pushed_at":"2026-02-22T17:23:16.000Z","size":913,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-23T03:59:48.952Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emredeveloper.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-17T19:47:23.000Z","updated_at":"2026-02-22T17:23:19.000Z","dependencies_parsed_at":"2026-02-22T00:00:34.041Z","dependency_job_id":null,"html_url":"https://github.com/emredeveloper/explainable-agent-lab","commit_stats":null,"previous_names":["emredeveloper/explainable-agent-lab"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/emredeveloper/explainable-agent-lab","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fexplainable-agent-lab","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fexplainable-agent-lab/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fexplainable-agent-lab/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fexplainable-agent-lab/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emredeveloper","download_url":"https://codeload.github.com/emredeveloper/explainable-agent-lab/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emredeveloper%2Fexplainable-agent-lab/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29768307,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T01:40:24.820Z","status":"online","status_checked_at":"2026-02-24T02:00:07.497Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-20T23:02:32.497Z","updated_at":"2026-05-03T17:04:55.662Z","avatar_url":"https://github.com/emredeveloper.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🔬 Explainable Agent Lab\n\n\u003e A local-first, explainable agent framework designed to guide developers in building robust AI agents.\n\nBuilding reliable agents is hard. LLMs hallucinate, get stuck in infinite loops, or fail to parse tools correctly. **Explainable Agent Lab** is built to solve this by focusing on **explainability and guidance**.\n\n✨ **Key Features:**\n- **Show the Hidden Errors:** Reveal exactly where and why an agent fails (e.g., low confidence, schema violations).\n- **Self-Healing:** The agent automatically analyzes its own errors and proposes alternative tool-based solutions.\n- **Visual Terminal Tracking:** Step-by-step interactive and colorful tracking using the `rich` library (`--verbose`).\n- **Detailed Diagnostic Reports:** Actionable suggestions on hallucination risks, loop patterns, and prompt improvements.\n- **Chaos Engineering (Stress Testing):** Inject simulated tool errors (e.g., timeouts, missing data) to test your agent's self-healing capabilities.\n- **Efficiency Diagnostics:** Track token usage and step counts to identify context window exhaustion and prompt inefficiencies.\n- **Multi-Agent Orchestration (Team of Thoughts):** Coordinate multiple specialized agents with transparent delegation plans, sub-agent traces, and orchestration diagnostics.\n\n---\n\n## 🚀 Quick Start\n\n### 1. Install\nInstall directly from PyPI:\n```bash\npip install explainable-agent\n```\n\nPyPI currently publishes `0.2.5` (released May 3, 2026). If PyPI is behind a future GitHub release, install the repository version instead:\n```bash\npip install \"git+https://github.com/emredeveloper/explainable-agent-lab.git@v0.2.5\"\n```\n\nFor development, clone the repo and run:\n```bash\npip install -e \".[dev]\"\n```\n\nCheck the installed package version with:\n```bash\npython -c \"import explainable_agent; print(explainable_agent.__version__)\"\n```\n\n### 2. Connect Your Local LLM\nYou can use any OpenAI-compatible local server like **Ollama** or **LM Studio**.\n\n- **Ollama:** `http://localhost:11434/v1` (e.g., model: `qwen3.5:9b`)\n- **LM Studio:** `http://localhost:1234/v1` (e.g., model: `google/gemma-3-12b`)\n\n*Tip: You can create a `.env` file in your working directory to set your defaults (see `.env.example`).*\n\n### 3. Run the Agent\nThe package installs a global CLI command `explainable-agent`.\n\n**Example using Ollama:**\n```bash\nexplainable-agent \\\n  --base-url http://localhost:11434/v1 \\\n  --model qwen3.5:9b \\\n  --task \"calculate_math: (215*4)-12\" \\\n  --verbose\n```\n\n---\n\n## 💻 Using the Python API\n\nEasily integrate the agent into your codebase or create custom tools using the `@define_tool` decorator.\n\nCheck out the `examples/` directory:\n- [`examples/basic_usage.py`](examples/basic_usage.py) - Small default smoke run using `.env`/CLI settings.\n- [`examples/custom_tool_usage.py`](examples/custom_tool_usage.py) - Register a custom Python tool and call it through the agent.\n- [`examples/showcase_all_features.py`](examples/showcase_all_features.py) - Compact local showcase for math, SQLite, custom tools, optional chaos mode, and optional JSONL eval.\n- [`examples/multi_agent_demo.py`](examples/multi_agent_demo.py) - Multi-agent orchestration with a researcher and SQLite specialist.\n\nRun quick examples with Ollama:\n```bash\npython examples/basic_usage.py \\\n  --base-url http://localhost:11434/v1 \\\n  --api-key ollama \\\n  --model qwen3.5:9b\n\npython examples/showcase_all_features.py \\\n  --base-url http://localhost:11434/v1 \\\n  --api-key ollama \\\n  --model qwen3.5:9b\n```\n\nUse `--include-sqlite`, `--include-custom`, `--include-chaos`, or `--include-eval` on the showcase when you want the slower optional scenarios.\n\n---\n\n## 📊 Evaluation \u0026 Custom Datasets\n\nEvaluate your fine-tuned models or custom datasets easily. The pipeline parses messy outputs, repairs broken JSON, and generates actionable Markdown reports.\n\n- **Custom JSONL datasets:**  \n  1. Create a `.jsonl` dataset (see `examples/custom_eval_sample.jsonl`).  \n  2. Run the evaluation:\n     ```bash\n     python scripts/eval_hf_tool_calls.py \\\n       --dataset examples/custom_eval_sample.jsonl \\\n       --model qwen3.5:9b\n     ```\n\n- **Built-in HF-style tool-calling sample (JSONL):**\n  A small complex function-calling benchmark is bundled under `data/evals/hf_complexfuncbench_first_turn_100.jsonl`.\n  Example with LM Studio and `google/gemma-3-12b`:\n  ```bash\n  python scripts/eval_hf_tool_calls.py \\\n    --dataset data/evals/hf_complexfuncbench_first_turn_100.jsonl \\\n    --base-url http://localhost:1234/v1 \\\n    --model google/gemma-3-12b \\\n    --limit 10 \\\n    --sampling head\n  ```\n\nWe also support standard benchmarks out of the box:\n- **HF Tool Calls:** `data/evals/hf_complexfuncbench_first_turn_100.jsonl`\n- **BFCL SQL:** `data/evals/bfcl_sql/BFCL_v3_sql.json`\n- **SWE-bench Lite:** `data/evals/swebench_lite_test.jsonl`\n\n---\n\n## 🔍 Tracing \u0026 Verbosity Modes\n\nThe agent supports two primary verbosity modes:\n\n- **Verbose mode (`verbose=True` or `--verbose`):**\n  - Prints an **Agent tools flow roadmap** at the start (task, model, config, available tools, and control flow).\n  - Shows rich, per-step panels including:\n    - Decision source (`model`, `explicit_request`, `heuristic_override`)\n    - Latency per step\n    - Rationale, confidence, tool name/input/output\n    - Error analysis and proposed fix (for self-healing steps)\n  - Ends with a **developer run summary** panel (tool flow recap, faithfulness note, efficiency diagnostics).\n\n- **Concise mode (`verbose=False`):**\n  - Prints a one-line **flow summary** (e.g., `Step 1: calculate_math [FAIL] -\u003e Step 2: calculate_math [OK] -\u003e Step 3: final_answer`).\n  - Shows total step count, self-healed error count, a short final answer preview, and key warnings (if any).\n\n---\n\n## 🛠️ Built-in Tools\nThe agent comes with out-of-the-box tools ready to use:\n`duckduckgo_search`, `calculate_math`, `read_text_file`, `list_workspace_files`, `now_utc`, `sqlite_init_demo`, `sqlite_list_tables`, `sqlite_describe_table`, `sqlite_query`, `sqlite_execute`.\n\n`duckduckgo_search` remains the tool name in the API, while the underlying search dependency is provided by `ddgs`.\n\n---\n*License: MIT | Current Release: v0.2.5*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femredeveloper%2Fexplainable-agent-lab","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femredeveloper%2Fexplainable-agent-lab","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femredeveloper%2Fexplainable-agent-lab/lists"}