{"id":22889912,"url":"https://github.com/phil65/llmling-agent","last_synced_at":"2025-12-16T04:19:26.715Z","repository":{"id":266673076,"uuid":"899003856","full_name":"phil65/llmling-agent","owner":"phil65","description":"Multi-agent workflows and complex Agent interactions, both via YAML manifest and programmatic usage. Pydantic-AI and LiteLLM backends. Human-in-the-loop integration.","archived":false,"fork":false,"pushed_at":"2025-05-31T17:12:11.000Z","size":4051,"stargazers_count":19,"open_issues_count":1,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-10T03:50:16.465Z","etag":null,"topics":["agent","ai","litellm","llm","llm-agent","pydantic","pydantic-ai","yaml"],"latest_commit_sha":null,"homepage":"https://phil65.github.io/llmling-agent/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phil65.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"phil65","custom":["https://www.paypal.me/phil65"]}},"created_at":"2024-12-05T12:50:17.000Z","updated_at":"2025-05-31T17:12:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"7fd1c4fc-7aff-48a0-805f-159e9698520a","html_url":"https://github.com/phil65/llmling-agent","commit_stats":null,"previous_names":["phil65/llmling-agent"],"tags_count":108,"template":false,"template_full_name":null,"purl":"pkg:github/phil65/llmling-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phil65%2Fllmling-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phil65%2Fllmling-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phil65%2Fllmling-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phil65%2Fllmling-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phil65","download_url":"https://codeload.github.com/phil65/llmling-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phil65%2Fllmling-agent/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265314438,"owners_count":23745270,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","ai","litellm","llm","llm-agent","pydantic","pydantic-ai","yaml"],"created_at":"2024-12-13T21:57:26.456Z","updated_at":"2025-12-16T04:19:26.700Z","avatar_url":"https://github.com/phil65.png","language":"Python","funding_links":["https://github.com/sponsors/phil65","https://www.paypal.me/phil65"],"categories":["Frameworks"],"sub_categories":["Advanced Components"],"readme":"# LLMling-Agent\n\n[![PyPI License](https://img.shields.io/pypi/l/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Package status](https://img.shields.io/pypi/status/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Monthly downloads](https://img.shields.io/pypi/dm/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Distribution format](https://img.shields.io/pypi/format/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Wheel availability](https://img.shields.io/pypi/wheel/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Python version](https://img.shields.io/pypi/pyversions/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Implementation](https://img.shields.io/pypi/implementation/llmling-agent.svg)](https://pypi.org/project/llmling-agent/)\n[![Releases](https://img.shields.io/github/downloads/phil65/llmling-agent/total.svg)](https://github.com/phil65/llmling-agent/releases)\n[![Github Contributors](https://img.shields.io/github/contributors/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/graphs/contributors)\n[![Github Discussions](https://img.shields.io/github/discussions/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/discussions)\n[![Github Forks](https://img.shields.io/github/forks/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/forks)\n[![Github Issues](https://img.shields.io/github/issues/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/issues)\n[![Github Watchers](https://img.shields.io/github/watchers/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/watchers)\n[![Github Stars](https://img.shields.io/github/stars/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/stars)\n[![Github last commit](https://img.shields.io/github/last-commit/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/commits)\n[![Github release date](https://img.shields.io/github/release-date/phil65/llmling-agent)](https://github.com/phil65/llmling-agent/releases)\n[![Github language count](https://img.shields.io/github/languages/count/phil65/llmling-agent)](https://github.com/phil65/llmling-agent)\n[![Github commits this year](https://img.shields.io/github/commit-activity/y/phil65/llmling-agent)](https://github.com/phil65/llmling-agent)\n[![Package status](https://codecov.io/gh/phil65/llmling-agent/branch/main/graph/badge.svg)](https://codecov.io/gh/phil65/llmling-agent/)\n[![PyUp](https://pyup.io/repos/github/phil65/llmling-agent/shield.svg)](https://pyup.io/repos/github/phil65/llmling-agent/)\n\n### [Read the documentation!](https://phil65.github.io/llmling-agent/)\n\n# 🚀 Getting Started\n\nLLMling Agent is a framework for creating and managing LLM-powered agents. It integrates with LLMling's resource system and provides structured interactions with language models.\n\n## ✨ Features\n\n- 🔄 Modern python written from ground up with Python 3.13\n- 🤝 Integrate multiple external ACP agents (Claude Code, Codex, Goose, etc.) into a single pool where they can cooperate on tasks\n- 🛡️ Complete (multi-)agent pool setup via YAML files including extensive JSON schema to help with creating configurations.\n- 🔌 Extensive MCP support including elicitation, sampling, progress reporting, multi-modality, including bridging to ACP / AG-UI protocols.\n- 💾 Storage providers to allow writing to local files, databases, etc. with many customizable backends. Log to SQL databases and pretty-print to a file according to your own wishes.\n- 🛜 Comletely UPath backed. All file operations (\u0026 Code execution) by agents are abstrated in a way that agents can operate directly on remote sources.\n- 📕 Integrated prompt management system.\n- 🔧 Tasks, tools, and what else you can expect from an Agent framework.\n- 👥 Easy human-in-the-loop interactions\n\n\n## Quick Start\n\n### Agent Client Protocol (ACP)\n\nThe fastest way to start chatting with an AI:\n\nllmling-agent supports the Agent Client Protocol for seamless integration with desktop applications and IDEs. Run your agents as ACP servers to enable bidirectional communication, session management, and file operations through JSON-RPC 2.0 over stdio.\n\n*The recommended client is Zed IDE (\u0026 soon Toad, a python client based on Textual)*\n\n```bash\n# Start ACP server\nllmling-agent serve-acp [path/to/config.yml]\n```\n\nCompatible with ACP-enabled Clients like Zed. See the [ACP Integration documentation](https://phil65.github.io/llmling-agent/advanced/acp_integration/) for setup instructions.\n\nRun `/help` in the chat to see what commands are at your disposal.\n\n### YAML configuration\n\nWhile you can define agents with 3 lines of YAML (or competely programmatic or via CLI),\nyou can also create agents as well as their connections, agent tasks, storage providers and much more via YAML.\nThis is the extended version\n\n```yaml\n# agents.yml\nagents:\n  analyzer:\n    name: \"Code Analyzer\"  # Display name\n    inherits: \"base_agent\"  # Optional parent config to inherit from\n    description: \"Code analysis specialist\"\n    debug: false\n    retries: 1  # Number of retries for failed operations\n    model:  # Model configuration\n      type: \"fallback\"  # Lot of special \"meta-models\" included out of the box!\n      models:  # Try models in sequence\n        - \"openai:gpt-5\"\n        - \"openai:gpt-5-nano\"\n        - \"anthropic:claude-sonnet-4-0\"\n    # Structured output\n    output_type:\n      type: \"inline\"  # or \"import\" for Python types\n      fields:\n        severity:\n          type: \"str\"\n          description: \"Issue severity\"\n        issues:\n          type: \"list[str]\"\n          description: \"Found issues\"\n\n    # Core behavior\n    system_prompts:\n      - \"You analyze code for potential issues and improvements.\"\n\n    # Session \u0026 History\n    session:\n      name: \"analysis_session\"\n      since: \"1h\"  # Only load messages from last hour\n      roles: [\"user\", \"assistant\"]  # Only specific message types\n\n    # Toolsets (available tool groups)\n    toolsets:\n      - type: agent_management  # Enables delegation\n      - type: resource_access   # Enables resource loading\n\n\n    # Knowledge sources\n    knowledge:\n      paths: [\"docs/**/*.md\"]  # Glob patterns for files\n      resources:\n        - type: \"repository\"\n          url: \"https://github.com/user/repo\"\n      prompts:\n        - type: \"file\"\n          path: \"prompts/analysis.txt\"\n\n    # MCP Server integration\n    mcp_servers:\n      - type: \"stdio\"\n        command: \"python\"\n        args: [\"-m\", \"mcp_server\"]\n        env:\n          DEBUG: \"1\"\n      - \"python -m other_server\"  # shorthand syntax\n\n    # Worker agents (specialists)\n    workers:\n      - type: agent\n        name: \"formatter\"\n        reset_history_on_run: true\n        pass_message_history: false\n      - \"linter\"  # shorthand syntax\n\n    # Message forwarding\n    connections:\n      - type: node\n        name: \"reporter\"\n        connection_type: \"run\"  # \"run\" | \"context\" | \"forward\"\n        priority: 1\n        queued: true\n        queue_strategy: \"latest\"\n        transform: \"my_module.transform_func\"\n        wait_for_completion: true\n        filter_condition:  # When to forward messages\n          type: \"word_match\"\n          words: [\"error\", \"warning\"]\n          case_sensitive: false\n        stop_condition:  # When to disconnect\n          type: \"message_count\"\n          max_messages: 100\n          count_mode: \"total\"  # or \"per_agent\"\n        exit_condition:  # When to exit application\n          type: \"cost_limit\"\n          max_cost: 10.0\n    # Event triggers\n    triggers:\n      - type: \"file\"\n        name: \"code_change\"\n        paths: [\"src/**/*.py\"]\n        extensions: [\".py\"]\n        debounce: 1000  # ms\nteams:\n  # Complex workflows via YAML\n  full_pipeline:\n    mode: sequential\n    members:\n      - analyzer\n      - planner\n    connections:\n      - type: node\n        name: final_reviewer\n        wait_for_completion: true\n      - type: file\n        path: \"reports/{date}_workflow.txt\"\n# Response type definitions\nresponses:\n  AnalysisResult:\n    response_schema:\n      type: \"inline\"\n      description: \"Code analysis result format\"\n      fields:\n        severity: {type: \"str\"}\n        issues: {type: \"list[str]\"}\n\n  ComplexResult:\n    type: \"import\"\n    import_path: \"myapp.types.ComplexResult\"\n\n# Storage configuration\nstorage:\n  providers:\n    - type: \"sql\"\n      url: \"sqlite:///history.db\"\n      pool_size: 5\n    - type: \"text_file\"\n      path: \"logs/chat.log\"\n      format: \"chronological\"\n  log_messages: true\n  log_conversations: true\n  log_commands: true\n\n# Pre-defined jobs\njobs:\n  analyze_code:\n    name: \"Code Analysis\"\n    description: \"Analyze code quality\"\n    prompt: \"Analyze this code: {code}\"\n    required_return_type: \"AnalysisResult\"\n    knowledge:\n      paths: [\"src/**/*.py\"]\n    tools: [\"analyze_complexity\", \"run_linter\"]\n```\n\nYou can use an Agents manifest in multiple ways:\n\n\n\n- Run it using the CLI\n\n```bash\nllmling-agent run --config agents.yml my_agent \"Some prompt\"\n```\n\n- Start *watch mode* and only react to triggers\n\n```bash\nllmling-agent watch --config agents.yml\n```\n\n\n### Agent Pool: Multi-Agent Coordination\n\nThe `AgentPool` allows multiple agents to work together on tasks, including external ACP-enabled agents like Claude Code, Codex, or Goose. Here's a practical example of parallel file downloading:\n\n```yaml\n# agents.yml\nagents:\n  file_getter:\n    model: openai:gpt-5-mini\n    toolsets:\n      - type: file_access  # includes download_file, read_file, list_directory\n    system_prompts:\n      - |\n        You are a download specialist. Just use the download_file tool\n        and report its results. No explanations needed.\n\n  overseer:\n    toolsets:\n      - type: agent_management  # Enables delegation and agent discovery tools\n    model: openai:gpt-5-mini\n    system_prompts:\n      - |\n        You coordinate downloads using available agents.\n        1. Check out the available agents and assign each of them the download task\n        2. Report the results.\n\n```\n\nProgrammatic Usage:\n\n```python\nfrom llmling_agent.delegation import AgentPool\n\nasync def main():\n    async with AgentPool(\"agents.yml\") as pool:\n        # first we create two agents based on the file_getter template\n        file_getter_1 = pool.get_agent(\"file_getter\")\n        file_getter_2 = pool.get_agent(\"file_getter\")\n        # then we form a team and execute the task\n        team = file_getter_1 \u0026 file_getter_2\n        responses = await team.run_parallel(\"Download https://example.com/file.zip\")\n\n        # Or let a coordinator orchestrate using his capabilities.\n        coordinator = pool.get_agent(\"coordinator\")\n        result = await overseer.run(\n            \"Download https://example.com/file.zip by delegating to all workers available!\"\n        )\n```\n\n#### External ACP Agents\n\nYou can also integrate external ACP-enabled agents into your pool via YAML configuration:\n\n```yaml\n# agents.yml\nacp_agents:\n  claude:\n    type: claude\n    display_name: \"Claude Code\"\n    description: \"Claude Code through ACP\"\n  goose:\n    type: goose\n    display_name: \"Goose\"\n    description: \"Block's Goose agent through ACP\"\n\nagents:\n  coordinator:\n    model: openai:gpt-5-mini\n    toolsets:\n      - type: agent_management  # Enables delegation to ACP agents\n```\n\n```python\nasync with AgentPool(\"agents.yml\") as pool:\n    # Access external ACP agents just like regular agents\n    claude = pool.get_agent(\"claude\")\n    result = await claude.run(\"Refactor this code\")\n```\n\nSee the [ACP Integration documentation](https://phil65.github.io/llmling-agent/advanced/acp_integration/#external-acp-agents) for supported agents and configuration options.\n\n\nThe framework provides three types of message nodes:\n\n1. **Agents**: Individual LLM-powered actors\n```python\n# Single agent processing\nanalyzer = pool.get_agent(\"analyzer\")\nresult = await analyzer.run(\"analyze this\")\n```\n\n2. **Teams**: Groups for parallel execution\n```python\n# Create team using \u0026 operator\nteam = analyzer \u0026 planner \u0026 executor\nresults = await team.run(\"handle this task\")\n```\n\n3. **TeamRuns**: Sequential execution chains\n```python\n# Create chain using | operator\nchain = analyzer | planner | executor\nresults = await chain.run(\"process in sequence\")\n```\n\nThe beauty of this system is that these nodes are completely composable:\n\n```python\n\ndef process_text(text: str) -\u003e str:\n    return text.upper()\n\n# Nested structures work naturally\nteam_1 = analyzer \u0026 planner  # Team\nteam_2 = validator \u0026 reporter  # Another team\nchain = team_1 | process_text | team_2  # Teams and Callables in a chain\n\n# Complex workflows become intuitive\n(analyzer \u0026 planner) | validator  # Team followed by validator\nteam_1 | (team_2 \u0026 agent_3)  # Chain with parallel components\n\n# Every node has the same core interface\nasync for message in node.run_iter(\"prompt\"):\n    print(message.content)\n\n# Monitoring works the same for all types\nprint(f\"Messages: {node.stats.message_count}\")\nprint(f\"Cost: ${node.stats.total_cost:.2f}\")\n```\n(note: the operator overloading is just syntactic sugar. In general, teams should be created\nusing `pool.create_team()` / `pool.create_team_run()` or `agent/team.connect_to()`)\n)\n\nAll message nodes support the same execution patterns:\n```python\n# Single execution\nresult = await node.run(\"prompt\")\n\n# Streaming\nasync for event in node.run_stream(\"prompt\"):\n    print(event)\n\n\n# Nested teams work naturally\nteam_1 = analyzer \u0026 planner  # First team\nteam_2 = validator \u0026 reporter  # Second team\nparallel_team = Team([team_1, agent_3, team_2])  # Team containing teams!\n\n# This means you can create sophisticated structures:\nresult = await parallel_team.run(\"analyze this\")  # Will execute:\n# - team_1 (analyzer \u0026 planner) in parallel\n# - agent_3 in parallel\n# - team_2 (validator \u0026 reporter) in parallel\n\n# And still use all the standard patterns:\nasync for msg in parallel_team.run_iter(\"prompt\"):\n    print(msg.content)\n\n# With full monitoring functionality:\nprint(f\"Total cost: ${parallel_team.stats.total_cost:.2f}\")\n\n```\n\nThis unified system makes it easy to:\n- Build complex workflows\n- Monitor message flow\n- Compose nodes in any combination\n- Use consistent patterns across all node types\n\nEach message in the system carries content, metadata, and execution information, providing a consistent interface across all types of interactions. See [Message System](docs/core-concepts/messages.md) for details.\n\n\n### Advanced Connection Features\n\nConnections between agents are highly configurable and support various patterns:\n\n```python\n# Basic connection in shorthand form.\nconnection = agent_a \u003e\u003e agent_b  # Forward all messages\n\n# Extended setup: Queued connection (manual processing)\nconnection = agent_a.connect_to(\n    agent_b,\n    queued=True,\n    queue_strategy=\"latest\",  # or \"concat\", \"buffer\"\n)\n# messages can queue up now\nawait connection.trigger(optional_additional_prompt)  # Process queued messages sequentially\n\n# Filtered connection (example: filter by keyword):\nconnection = agent_a.connect_to(\n    agent_b,\n    filter_condition=lambda ctx: \"keyword\" in ctx.message.content,\n)\n\n# Conditional disconnection (example: disconnect after cost limit):\nconnection = agent_a.connect_to(\n    agent_b,\n    filter_condition=lambda ctx: ctx.stats.total_cost \u003e 1.0,\n)\n\n# Message transformations\nasync def transform_message(message: str) -\u003e str:\n    return f\"Transformed: {message}\"\n\nconnection = agent_a.connect_to(agent_b, transform=transform_message)\n\n# Connection statistics\nprint(f\"Messages processed: {connection.stats.message_count}\")\nprint(f\"Total tokens: {connection.stats.token_count}\")\nprint(f\"Total cost: ${connection.stats.total_cost:.2f}\")\n```\n\nThe two basic programmatic patterns of this librry are:\n\n1. Tree-like workflows (hierarchical):\n```python\n# Can be modeled purely with teams/chains using \u0026 and |\nteam_a = agent1 \u0026 agent2  # Parallel branch 1\nteam_b = agent3 \u0026 agent4  # Parallel branch 2\nchain = preprocessor | team_a | postprocessor  # Sequential with team\nnested = Team([chain, team_b])  # Hierarchical nesting\n```\n\n2. DAG (Directed Acyclic Graph) workflows:\n```python\n# Needs explicit signal connections for non-tree patterns\nanalyzer = Agent(\"analyzer\")\nplanner = Agent(\"planner\")\nexecutor = Agent(\"executor\")\nvalidator = Agent(\"validator\")\n\n# Can't model this with just teams - need explicit connections\nanalyzer.connect_to(planner)\nanalyzer.connect_to(executor)  # Same source to multiple targets\nplanner.connect_to(validator)\nexecutor.connect_to(validator) # Multiple sources to same target\nvalidator.connect_to(executor) # Cyclic connections\n```\n\nBOTH connection types can be set up for BOTH teams and agents intiuiviely in the YAML file.\n\nYou can also use LLMling-models for more sophisticated human-in-the-loop integration:\n- Remote human operators via network\n- Hybrid human-AI workflows\n- Input streaming support\n- Custom UI integration\n\n\n### Multi-Modal Support\n\nHandle images and PDFs alongside text (depends on provider / model support)\n\n```python\nfrom llmling_agent import Agent\n\nasync with Agent(...) as agent:\n    result = await agent.run(\"What's in this image?\", pathlib.Path(\"image.jpg\"))\n    result = await agent.run(\"What's in this PDF?\", pathlib.Path(\"document.pdf\"))\n```\n\n### Command System\n\nExtensive slash commands available when used via ACP:\n\n```bash\n/list-tools              # Show available tools\n/enable-tool tool_name   # Enable specific tool\n/connect other_agent     # Forward results\n/model gpt-5            # Switch models\n/history search \"query\"  # Search conversation\n/stats                   # Show usage statistics\n```\n\n### Storage \u0026 Analytics\n\nAll interaction is tracked using (multiple) configurable storage providers.\nInformation can get fetched  via CLI.\n\n\n```bash\n# View recent conversations\nllmling-agent history show\nllmling-agent history show --period 24h  # Last 24 hours\nllmling-agent history show --query \"database\"  # Search content\n\n# View usage statistics\nllmling-agent history stats  # Basic stats\nllmling-agent history stats --group-by model  # Model usage\nllmling-agent history stats --group-by day    # Daily breakdown\n```\n\n## 📚 MkDocs Integration\n\nIn combination with [MkNodes](https://github.com/phil65/mknodes) and the [MkDocs plugin](https://github.com/phil65/mkdocs_mknodes),\nyou can easily generate static documentation for websites with a few lines of code.\n\n```python\n\n@nav.route.page(\"Feature XYZ\", icon=\"oui:documentation\", hide=\"toc\")\ndef gen_docs(page: mk.MkPage):\n    \"\"\"Generate docs using agents.\"\"\"\n    agent = Agent(model=\"openai:gpt-5-nano\")\n    page += mk.MkAdmonition(\"MkNodes includes all kinds of Markdown objects to generate docs!\")\n    source_code = load_source_code_from_folder(...)\n    page += mk.MkCode() # if you want to display source code\n    result = agent.run.sync(\"Describle Feature XYZ in MkDocs compatible markdown including examples.\", content)\n    page += result.content\n```\n\n### [Read the documentation for further info!](https://phil65.github.io/llmling-agent/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphil65%2Fllmling-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphil65%2Fllmling-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphil65%2Fllmling-agent/lists"}