{"id":35094365,"url":"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook","last_synced_at":"2026-01-18T17:00:41.186Z","repository":{"id":313717733,"uuid":"1052387869","full_name":"oxbshw/LLM-Agents-Ecosystem-Handbook","owner":"oxbshw","description":"One-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials, ecosystem guides, and evaluation tools.","archived":false,"fork":false,"pushed_at":"2025-09-08T01:59:44.000Z","size":1803,"stargazers_count":23,"open_issues_count":0,"forks_count":6,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-08T03:30:27.192Z","etag":null,"topics":["ai","ai-agent","ai-agents","fine-tuning","finetuning-llms","freamework","llm","llmops","local-development","mcp-server","memory","rag","rag-chatbot","voice-agent"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxbshw.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-08T01:24:26.000Z","updated_at":"2025-09-08T03:28:36.000Z","dependencies_parsed_at":"2025-09-08T03:30:33.151Z","dependency_job_id":null,"html_url":"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook","commit_stats":null,"previous_names":["oxbshw/llm-agents-ecosystem-handbook"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/oxbshw/LLM-Agents-Ecosystem-Handbook","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxbshw%2FLLM-Agents-Ecosystem-Handbook","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxbshw%2FLLM-Agents-Ecosystem-Handbook/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxbshw%2FLLM-Agents-Ecosystem-Handbook/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxbshw%2FLLM-Agents-Ecosystem-Handbook/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxbshw","download_url":"https://codeload.github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxbshw%2FLLM-Agents-Ecosystem-Handbook/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28543490,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T14:59:57.589Z","status":"ssl_error","status_checked_at":"2026-01-18T14:59:46.540Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-agent","ai-agents","fine-tuning","finetuning-llms","freamework","llm","llmops","local-development","mcp-server","memory","rag","rag-chatbot","voice-agent"],"created_at":"2025-12-27T15:04:24.315Z","updated_at":"2026-01-18T17:00:41.176Z","avatar_url":"https://github.com/oxbshw.png","language":"Python","funding_links":[],"categories":["📚 Projects (1974 total)","Tutorials"],"sub_categories":["Documentation","Agents"],"readme":"# LLM Agents \u0026 Ecosystem Handbook\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://awesome.re\"\u003e\u003cimg src=\"https://awesome.re/badge.svg\" alt=\"Awesome\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/stargazers\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/oxbshw/LLM-Agents-Ecosystem-Handbook?style=social\" alt=\"GitHub stars\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/issues\"\u003e\u003cimg src=\"https://img.shields.io/github/issues/oxbshw/LLM-Agents-Ecosystem-Handbook\" alt=\"Issues\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/pulls\"\u003e\u003cimg src=\"https://img.shields.io/github/issues-pr/oxbshw/LLM-Agents-Ecosystem-Handbook\" alt=\"Open PRs\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/graphs/contributors\"\u003e\u003cimg src=\"https://img.shields.io/github/contributors/oxbshw/LLM-Agents-Ecosystem-Handbook\" alt=\"Contributors\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-green.svg\" alt=\"MIT License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/commits/main\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/oxbshw/LLM-Agents-Ecosystem-Handbook\" alt=\"Last commit\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003eA unified handbook for building, deploying and understanding LLM agents and the wider ecosystem\u003c/strong\u003e\u003c/p\u003e\n\nA polished, curated collection of **Large Language Model (LLM) agents**, tutorials and ecosystem insights. This handbook highlights projects that push the boundaries of generative AI, multi-agent collaboration, retrieval-augmented generation (RAG), voice and game agents, and more. It goes beyond simple link aggregation, aiming to be a one-stop reference for building, deploying, and understanding LLM applications across the entire stack.\n\n\u003e **Tip:** If you enjoy this list, please consider starring the repository to help others discover it!\n\n---\n\n## Table of Contents\n\n- [Top Agent Frameworks](#top-agent-frameworks)\n- [Agent Toolkits \u0026 Platforms](#agent-toolkits--platforms)\n- [Starter AI Agents](#starter-ai-agents)\n- [Advanced AI \u0026 Domain-Specific Agents](#advanced-ai--domain-specific-agents)\n- [Multi-Agent Teams](#multi-agent-teams)\n- [Voice \u0026 Game Agents](#voice--game-agents)\n- [RAG \u0026 Memory Examples](#rag--memory-examples)\n- [MCP Agent Integrations](#mcp-agent-integrations)\n- [LLM Evaluation Frameworks](#llm-evaluation-frameworks)\n- [Example Projects](#example-projects)\n- [Tutorials \u0026 Learning Resources](#tutorials--learning-resources)\n  - [RAG Tutorials](#rag-tutorials)\n  - [Memory Apps Tutorials](#memory-apps-tutorials)\n  - [Chat with X Tutorials](#chat-with-x-tutorials)\n  - [LLM Fine-Tuning Tutorials](#llm-fine-tuning-tutorials)\n- [Other Educational Spaces](#other-educational-spaces)\n- [Unique Features](#unique-features)\n- [Languages \u0026 Multilingual Support](#languages--multilingual-support)\n- [Interactive Demos \u0026 Resources](#interactive-demos--resources)\n  - [Web Apps](#web-apps)\n  - [Jupyter Notebooks](#jupyter-notebooks)\n- [Datasets \u0026 Design Assets](#datasets--design-assets)\n- [Documentation \u0026 Roadmap](#documentation--roadmap)\n- [Complete Applications](#complete-applications)\n- [Beginner’s Guide](#beginners-guide)\n- [Contributing](#contributing)\n- [License](#license)\n- [Maintainer](#maintainer)\n\n---\n\n## Why this repository stands out\n\nThis curated collection aims to be a **comprehensive resource** for developers and researchers building their own LLM applications. In addition to code examples, it provides:\n\n- **Comparative analysis of leading agent frameworks:** A quick matrix contrasting frameworks (LangGraph, AutoGen, CrewAI, Smolagents, etc.) with key features to help you choose.\n- **Guidance on framework selection:** Practical advice based on task complexity, collaboration needs and ecosystem integrations.\n- **LLM evaluation toolbox:** Summaries of tools like Promptfoo, DeepEval, MLflow LLM Evaluate, RAGAs and Langfuse to measure performance and safety.\n- **60+ skeleton projects:** The `agents` folder contains scaffolded agents across many domains (blogging, medical imaging, music generation, multimodal input, news, finance, research, scraping, consultancy, system design, compliance, marketing, scheduling, supply-chain, healthcare, education). Each skeleton includes a `README.md` and `main.py`.\n- **Agent skeleton generator:** [`scripts/create_agent.py`](scripts/create_agent.py) to spin up new agent skeletons in seconds.\n\n---\n\n## Top Agent Frameworks\n\n| Framework | Description \u0026 Key Features |\n|---|---|\n| **LangGraph** | Graph/DAG-based orchestration for complex multi-step workflows. |\n| **OpenAI Agents SDK** | Structured runtime with tool-calling and role-based agents. |\n| **AutoGen (AG2)** | Event-driven multi-agent conversations and human-in-the-loop. |\n| **CrewAI** | Role-based “crew” collaboration with memory and error handling. |\n| **Google AgentKit (ADK)** | Modular Gemini/Vertex AI agent kit with hierarchical tools. |\n| **Dify** | Low-code builder with RAG and function calling. |\n| **LangChain \u0026 Tools** | Mature chains, memory and 3rd-party integrations. |\n| **Smolagents** | Minimal, code-centric loop (agents write \u0026 execute code). |\n| **Semantic Kernel** | .NET-first skills/plans; enterprise-friendly. |\n| **LlamaIndex Agents** | Retrieval-focused agents for data-heavy apps. |\n| **Strands Agents** | Provider-agnostic SDK with OpenTelemetry. |\n| **Pydantic AI** | Type-safe IO/tool signatures with great DX. |\n\n---\n\n## Agent Toolkits \u0026 Platforms\n\n| Project | Description |\n|---|---|\n| **AutoGPT** | Toolkit for autonomous agents (creation, benchmarking, UI/CLI). |\n| **Ollama** | Run LLMs locally across macOS/Windows/Linux/Docker. |\n| **Lobe Chat** | Open-source chat UI with plugins and multimodal support. |\n| **OpenDevin** | Open initiative towards an AI software engineer. |\n| **Open Interpreter** | Natural-language coding \u0026 local computer control. |\n| **MetaGPT** | Multi-agent “virtual company” for complex tasks. |\n| **PrivateGPT** | Secure offline Q\u0026A over your documents. |\n| **GPT-Engineer** | From natural-language spec to code. |\n| **LlamaIndex Tools** | Connectors/tools for data agents. |\n| **Flowise** | Drag-and-drop builder for LLM workflows. |\n| **FastChat** | Train/serve/evaluate chatbots. |\n| **Mem0** | Memory layer for personalised LLMs. |\n| **Cal.ai** | Scheduling assistant with email handling. |\n| **Aider** | CLI pair-programming agent with Git integration. |\n| **Jan** | Offline ChatGPT-style desktop app. |\n\n---\n\n## Starter AI Agents\n\n| Agent | Description |\n|---|---|\n| **AI Blog to Podcast Agent** | Convert blog posts into podcasts. |\n| **AI Data Analysis Agent** | Insights from CSV/structured data. |\n| **AI Travel Agent** | Trip itineraries (local/cloud). |\n| **AI Music Generator** | Compose via generative models. |\n| **AI Meme Generator (Browser)** | Creates memes by overlaying captions on images. |\n| **AI Breakup Recovery Agent** | Supportive advice for emotional situations. |\n| **AI Health \u0026 Fitness Agent** | Health metrics \u0026 coaching. |\n| **Gemini Multimodal Agent** | Text+image multimodal demo. |\n\n---\n\n## Advanced AI \u0026 Domain-Specific Agents\n\n| Agent | Description |\n|---|---|\n| **AI Deep Research Agent** | Multi-source research \u0026 synthesis. |\n| **AI Consultant Agent** | Domain-expert strategy \u0026 advice. |\n| **AI System Architect Agent** | From requirements to architecture. |\n| **AI Lead Generation Agent** | Identify \u0026 qualify prospects. |\n| **AI Meeting Agent** | Summaries \u0026 action items. |\n| **OpenAI Research Agent** | Research workflows with tools. |\n| **Explainable AI Finance Agent** | Finance with interpretability. |\n| **Web Scraping Agent** | Crawl \u0026 extract structured data. |\n| **Document Processing Agent** | OCR + analysis/summarisation. |\n| **Sentiment Analysis Agent** | Classify sentiment at scale. |\n| **Technical Translation Agent** | Preserve domain terminology. |\n| **Research Synthesizer Agent** | RAG + coherent reporting. |\n\n---\n\n## Multi-Agent Teams\n\n| Team | Description |\n|---|---|\n| **Competitor Intelligence Team** | Market/competitor research \u0026 reporting. |\n| **Finance Agent Team** | Budgeting, forecasting and reporting. |\n| **Teaching Agent Team** | Lesson planning, delivery and assessment. |\n| **Multi-Agent Team Demo** | Cross-role collaboration patterns. |\n| **Mixture of Agents Demo** | Specialised agent ensembles. |\n\n---\n\n## Voice \u0026 Game Agents\n\n| Agent | Description |\n|---|---|\n| **Voice Summary Agent** | Transcribe and summarise audio. |\n| **AI Audio Tour Agent** | Generate audio tours for museums/cities. |\n| **Customer Support Voice Agent** | Handle spoken queries and log issues. |\n| **Voice RAG Agent** | Voice input + retrieval + TTS. |\n| **Tic-Tac-Toe Agent** | Autonomous gameplay template. |\n\n---\n\n## RAG \u0026 Memory Examples\n\n| Example | Description |\n|---|---|\n| **Agentic RAG with Reasoning** | Retrieval → reasoning → generation. |\n| **Hybrid Search RAG** | Vector + keyword retrieval. |\n| **Vision RAG** | Apply RAG to visual data. |\n| **CRAG (Corrective RAG)** | Human-feedback corrective loop. |\n| **Local RAG Agent** | Fully offline retrieval pipeline. |\n\n---\n\n## MCP Agent Integrations\n\n| Agent | Description |\n|---|---|\n| **Browser MCP Agent** | Drive a browser (search/click/forms). |\n| **GitHub MCP Agent** | Read/write/manage repositories. |\n| **Notion MCP Agent** | Create/update/query Notion pages \u0026 DBs. |\n| **Travel Planner MCP Agent Team** | Multi-agent trip planning via MCP. |\n\n---\n\n## LLM Evaluation Frameworks\n\nSee [`evaluation_frameworks/README.md`](evaluation_frameworks/README.md) for details about Promptfoo, DeepEval, MLflow LLM Evaluate, RAGAs, Deepchecks, LangSmith, TruLens, Arize Phoenix and Langfuse.\n\n---\n\n## Example Projects\n\nThe `agents` directory contains many **agent skeletons** organised by category. Category folders (`starter`, `advanced`, `teams`, `rag`) act as **indexes**, while each agent lives in its own top-level folder under `agents/`.\n\n### Quick picks\n\n- **Summarization Agent** – [agents/summarization_agent](agents/summarization_agent)  \n- **Data Analysis Agent** – [agents/data_analysis_agent](agents/data_analysis_agent)  \n- **Travel Itinerary Agent** – [agents/travel_itinerary_agent](agents/travel_itinerary_agent)  \n- **Voice Assistant Demo** – [agents/voice_agent_demo](agents/voice_agent_demo)  \n- **Meme Generator Agent** – [agents/meme_generator_agent](agents/meme_generator_agent)  \n- **Health \u0026 Fitness Agent** – [agents/health_fitness_agent](agents/health_fitness_agent)  \n- **Breakup Recovery Agent** – [agents/breakup_recovery_agent](agents/breakup_recovery_agent)  \n- **AI Blog to Podcast Agent** – [agents/ai_blog_to_podcast_agent](agents/ai_blog_to_podcast_agent)  \n- **AI Medical Imaging Agent** – [agents/ai_medical_imaging_agent](agents/ai_medical_imaging_agent)  \n- **AI Music Generator Agent** – [agents/ai_music_generator_agent](agents/ai_music_generator_agent)  \n- **Local News Agent** – [agents/local_news_agent](agents/local_news_agent)  \n- **Gemini Multimodal Demo** – [agents/gemini_multimodal_agent_demo](agents/gemini_multimodal_agent_demo)\n\n---\n\n## Tutorials \u0026 Learning Resources\n\nHands-on tutorials live in [`tutorials/`](tutorials) directory.\n\n- **RAG Tutorials** → [`tutorials/rag_tutorials`](tutorials/rag_tutorials)  \n- **Memory Apps Tutorials** → [`tutorials/memory_apps`](tutorials/memory_apps)  \n- **Chat with X Tutorials** → [`tutorials/chat_with_x_tutorials`](tutorials/chat_with_x_tutorials)  \n- **Fine-Tuning Tutorials** → [`tutorials/fine_tuning_tutorials`](tutorials/fine_tuning_tutorials)\n\n---\n\n## Other Educational Spaces\n\n- **Interactive demos \u0026 notebooks:** [`web_apps`](web_apps), [`notebooks`](notebooks)  \n- **Datasets \u0026 design assets:** [`datasets`](datasets), [`design`](design)  \n- **LLM ecosystem overview:** [`ecosystem/overview.md`](ecosystem/overview.md)  \n- **Complete applications:** [`complete_apps`](complete_apps)\n\n---\n\n## Unique Features\n\n- **Educational focus:** Detailed tutorials (RAG, memory, chat with X, fine-tuning) + 60+ scaffolded agents.  \n- **Framework comparison \u0026 guidance:** Practical, vendor-neutral advice.  \n- **Agent skeleton generator:** `scripts/create_agent.py`.  \n- **Evaluation toolbox:** Promptfoo, DeepEval, RAGAs, etc.  \n- **Ecosystem breadth:** Training, tools, production, local inference, operations, interpretability.  \n- **Community roadmap:** See [`docs/roadmap.md`](docs/roadmap.md).  \n\n---\n\n## Languages \u0026 Multilingual Support\n\nWe welcome translations. See [`TRANSLATION.md`](TRANSLATION.md). A **Technical Translation Agent** exists under `agents/technical_translation_agent`.\n\n---\n\n## Interactive Demos \u0026 Resources\n\n### Web Apps\n\n- **Streamlit Summariser** – [`web_apps/streamlit_summarizer`](web_apps/streamlit_summarizer)  \n- **Gradio FAQ Bot** – [`web_apps/gradio_faq_bot`](web_apps/gradio_faq_bot)\n\n### Jupyter Notebooks\n\n- **Getting Started** – [`notebooks/getting_started.ipynb`](notebooks/getting_started.ipynb)\n\n---\n\n## Datasets \u0026 Design Assets\n\n- **Sample dataset** – [`datasets/sample_products.csv`](datasets/sample_products.csv) (+ [`datasets/README.md`](datasets/README.md))  \n- **Architecture diagram** – [`design/architecture_diagram.png`](design/architecture_diagram.png) (+ [`design/README.md`](design/README.md))\n\n---\n\n## Documentation \u0026 Roadmap\n\n- **Best practices** – [`docs/best_practices.md`](docs/best_practices.md)  \n- **Framework comparison** – [`docs/framework_comparison.md`](docs/framework_comparison.md)  \n- **Evaluation frameworks guide** – [`evaluation_frameworks/README.md`](evaluation_frameworks/README.md)  \n- **Quickstart** – [`tutorials/quickstart.md`](tutorials/quickstart.md)  \n- **Roadmap** – [`docs/roadmap.md`](docs/roadmap.md)  \n- **Changelog** – [`CHANGELOG.md`](CHANGELOG.md)\n\n---\n\n## Complete Applications\n\n- **Task Planner** – [`complete_apps/task_planner`](complete_apps/task_planner)  \n- **Health Coach** – [`complete_apps/health_coach`](complete_apps/health_coach)\n\n---\n\n## Beginner’s Guide\n\nIf you’re new to LLMs, start here → [`docs/beginners_guide.md`](docs/beginners_guide.md)\n\n---\n\n## Contributing\n\nContributions are welcome! Please open an issue or PR. Ensure new entries are permissively licensed (MIT/Apache-2.0) and well-documented.\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n\n---\n\n## Maintainer\n\nCurated \u0026 maintained by **Sayed Allam** ([oxbshw](https://github.com/oxbshw)). If you find this helpful, please ⭐ star the repo and share feedback via issues/PRs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxbshw%2FLLM-Agents-Ecosystem-Handbook","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxbshw%2FLLM-Agents-Ecosystem-Handbook","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxbshw%2FLLM-Agents-Ecosystem-Handbook/lists"}