{"id":34251758,"url":"https://github.com/flowllm-ai/flowllm","last_synced_at":"2025-12-16T10:03:57.374Z","repository":{"id":319687805,"uuid":"1036560368","full_name":"FlowLLM-AI/flowllm","owner":"FlowLLM-AI","description":"FlowLLM: Simplifying LLM-based HTTP/MCP Service Development","archived":false,"fork":false,"pushed_at":"2025-12-15T07:01:08.000Z","size":4523,"stargazers_count":20,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-15T08:22:18.110Z","etag":null,"topics":["agent","http-server","llm","llm-workflow","mcp-server","sharing"],"latest_commit_sha":null,"homepage":"https://flowllm-ai.github.io/flowllm/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FlowLLM-AI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-12T08:54:42.000Z","updated_at":"2025-12-15T07:01:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"f11df349-c416-4f52-b425-24a839055ea4","html_url":"https://github.com/FlowLLM-AI/flowllm","commit_stats":null,"previous_names":["flowllm-ai/flowllm"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/FlowLLM-AI/flowllm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FlowLLM-AI%2Fflowllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FlowLLM-AI%2Fflowllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FlowLLM-AI%2Fflowllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FlowLLM-AI%2Fflowllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FlowLLM-AI","download_url":"https://codeload.github.com/FlowLLM-AI/flowllm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FlowLLM-AI%2Fflowllm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27762497,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-16T02:00:10.477Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","http-server","llm","llm-workflow","mcp-server","sharing"],"created_at":"2025-12-16T10:03:54.510Z","updated_at":"2025-12-16T10:03:57.363Z","avatar_url":"https://github.com/FlowLLM-AI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/figure/logo.png\" alt=\"FlowLLM Logo\" width=\"50%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eFlowLLM: Simplifying LLM-based HTTP/MCP Service Development\u003c/strong\u003e\u003cbr\u003e\n  \u003cem\u003e\u003csub\u003eIf you find it useful, please give us a ⭐ Star. Your support drives our continuous improvement.\u003c/sub\u003e\u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/flowllm/\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.10+-blue\" alt=\"Python Version\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/flowllm/\"\u003e\u003cimg src=\"https://img.shields.io/badge/pypi-0.2.0.0-blue?logo=pypi\" alt=\"PyPI Version\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-Apache--2.0-black\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/flowllm-ai/flowllm\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/flowllm-ai/flowllm?style=social\" alt=\"GitHub Stars\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  English | \u003ca href=\"./README_ZH.md\"\u003e简体中文\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## 📖 Introduction\n\nFlowLLM encapsulates LLM, Embedding, and vector_store capabilities as HTTP/MCP services. It is suitable for AI assistants, RAG applications, and workflow services, and can be integrated into MCP-compatible client tools.\n\n### 🏗️ Architecture Overview\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/figure/framework.png\" alt=\"FlowLLM Framework\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n### 🌟 Applications Based on FlowLLM\n\n| Project Name                                  | Description                          |\n|-----------------------------------------------|--------------------------------------|\n| [ReMe](https://github.com/agentscope-ai/ReMe) | Memory management toolkit for agents |\n\n### 📢 Recent Updates\n\n| Date       | Update Content                                                                                                                                                                                                                                                            |\n|------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| 2025-11-15 | Added [File Tool Op](docs/zh/guide/file_tool_op_guide.md) feature with 13 file operation tools, supporting file reading, writing, editing, searching, directory operations, system command execution, and task management                                                 |\n| 2025-11-14 | Added Token counting capability, supporting accurate calculation of token counts for messages and tools via `self.token_count()` method, with support for multiple backends (base, openai, hf). See configuration examples in [default.yaml](flowllm/config/default.yaml) |\n\n### 📚 Learning Resources\n\nProject developers will share their latest learning materials here.\n\n| Date       | Title                                                                                                  | Description                                                                       |\n|------------|--------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|\n| 2025-11-24 | [Mem-PAL: Memory-Augmented Personalized Assistant](./docs/zh/reading/20251124-mem-pal.md)              | Mem-PAL: Memory-Augmented Personalized Assistant with Log-based Structured Memory |\n| 2025-11-14 | [HaluMem Analysis](./docs/zh/reading/20251114-halumem.md)                                              | HaluMem: Evaluating Hallucinations in Memory Systems of Agents Analysis           |\n| 2025-11-13 | [Gemini CLI Context Management Mechanism](./docs/zh/reading/20251113-gemini-cli-context-management.md) | Multi-layer Context Management Strategy for Gemini CLI                            |\n| 2025-11-10 | [Context Management Guide](./docs/zh/reading/20251110-manus-context-report.md)                         | Context Management Guide                                                          |\n| 2025-11-10 | [LangChain\u0026Manus Video Materials](./docs/zh/reading/20251110-manus-context-raw.md)                     | LangChain \u0026 Manus Context Management Video                                        |\n\n### ⭐ Core Features\n\n- **Simple Op Development**: Inherit from `BaseOp` or `BaseAsyncOp` and implement your business logic. FlowLLM provides lazy-initialized LLM, Embedding models, and vector stores accessible via `self.llm`, `self.embedding_model`, and `self.vector_store`. It also offers prompt template management through `prompt_format()` and `get_prompt()` methods. Additionally, FlowLLM includes built-in token counting capabilities. Use `self.token_count()` to accurately calculate token counts for messages and tools, supporting multiple backends (base, openai, hf, etc.).\n\n- **Flexible Flow Orchestration**: Compose Ops into Flows via YAML configuration. `\u003e\u003e` denotes serial composition; `|` denotes parallel composition. For example, `SearchOp() \u003e\u003e (AnalyzeOp() | TranslateOp()) \u003e\u003e FormatOp()` builds complex workflows. Define input/output schemas and start the service with `flowllm config=your_config`.\n\n- **Automatic Service Generation**: FlowLLM automatically generates HTTP, MCP, and CMD services. The HTTP service provides RESTful APIs with synchronous JSON and HTTP Stream responses. The MCP service registers as Model Context Protocol tools for MCP-compatible clients. The CMD service executes a single Op in command-line mode for quick testing and debugging.\n\n---\n\n## ⚡ Quick Start\n\n### 📦 Step0 Installation\n\n#### 📥 From PyPI\n\n```bash\npip install flowllm\n```\n\n#### 🔧 From Source\n\n```bash\ngit clone https://github.com/flowllm-ai/flowllm.git\ncd flowllm\npip install -e .\n```\n\nFor detailed installation and configuration, refer to the [Installation Guide](docs/zh/guide/installation.md).\n\n### ⚙️ Configuration\n\nCreate a `.env` file and configure your API keys. Copy from `example.env` and modify:\n\n```bash\ncp example.env .env\n```\n\nConfigure your API keys in the `.env` file:\n\n```bash\nFLOW_LLM_API_KEY=sk-xxxx\nFLOW_LLM_BASE_URL=https://xxxx/v1\nFLOW_EMBEDDING_API_KEY=sk-xxxx\nFLOW_EMBEDDING_BASE_URL=https://xxxx/v1\n```\n\nFor detailed configuration, refer to the [Configuration Guide](docs/zh/guide/config_guide.md).\n\n### 🛠️ Step1 Build Op\n\n```python\nfrom flowllm.core.context import C\nfrom flowllm.core.op import BaseAsyncOp\nfrom flowllm.core.schema import Message\nfrom flowllm.core.enumeration import Role\n\n@C.register_op()\nclass SimpleChatOp(BaseAsyncOp):\n    async def async_execute(self):\n        query = self.context.get(\"query\", \"\")\n        messages = [Message(role=Role.USER, content=query)]\n\n        # Use token_count method to calculate token count\n        token_num = self.token_count(messages)\n        print(f\"Input tokens: {token_num}\")\n\n        response = await self.llm.achat(messages=messages)\n        self.context.response.answer = response.content.strip()\n```\n\nFor details, refer to the [Simple Op Guide](docs/zh/guide/async_op_minimal_guide.md), [LLM Op Guide](docs/zh/guide/async_op_llm_guide.md), and [Advanced Op Guide](docs/zh/guide/async_op_advance_guide.md) (including Embedding, VectorStore, and concurrent execution).\n\n### 📝 Step2 Configure Config\n\nThe following example demonstrates building an MCP (Model Context Protocol) service. Create a configuration file `my_mcp_config.yaml`:\n\n```yaml\nbackend: mcp\n\nmcp:\n  transport: sse\n  host: \"0.0.0.0\"\n  port: 8001\n\nflow:\n  demo_mcp_flow:\n    flow_content: MockSearchOp()\n    description: \"Search results for a given query.\"\n    input_schema:\n      query:\n        type: string\n        description: \"User query\"\n        required: true\n\nllm:\n  default:\n    backend: openai_compatible\n    model_name: qwen3-30b-a3b-instruct-2507\n    params:\n      temperature: 0.6\n    token_count: # Optional, configure token counting backend\n      model_name: Qwen/Qwen3-30B-A3B-Instruct-2507\n      backend: hf  # Supports base, openai, hf, etc.\n      params:\n        use_mirror: true\n```\n\n### 🚀 Step3 Start MCP Service\n\n```bash\nflowllm \\\n  config=my_mcp_config \\\n  backend=mcp \\  # Optional, overrides config\n  mcp.transport=sse \\  # Optional, overrides config\n  mcp.port=8001 \\  # Optional, overrides config\n  llm.default.model_name=qwen3-30b-a3b-thinking-2507  # Optional, overrides config\n```\n\nAfter the service starts, refer to the [Client Guide](docs/zh/guide/client_guide.md) to use the service and obtain the tool_call required by the model.\n\n---\n\n## 📚 Detailed Documentation\n\n### 🚀 Getting Started\n- [Installation Guide](docs/zh/guide/installation.md)\n- [Configuration Guide](docs/zh/guide/config_guide.md)\n\n### 🔧 Op Development\n- [Op Introduction](docs/zh/guide/op_introduction.md)\n- [Simple Op Guide](docs/zh/guide/async_op_minimal_guide.md)\n- [LLM Op Guide](docs/zh/guide/async_op_llm_guide.md)\n- [Advanced Op Guide](docs/zh/guide/async_op_advance_guide.md)\n- [Tool Op Guide](docs/zh/guide/async_tool_op_guide.md)\n- [File Tool Op Guide](docs/zh/guide/file_tool_op_guide.md)\n- [Vector Store Guide](docs/zh/guide/vector_store_guide.md)\n\n### 🔀 Flow Orchestration\n- [Flow Guide](docs/zh/guide/flow_guide.md)\n\n### 🌐 Service Usage\n- [HTTP Service Guide](docs/zh/guide/http_service_guide.md)\n- [HTTP Stream Guide](docs/zh/guide/http_stream_guide.md)\n- [MCP Service Guide](docs/zh/guide/mcp_service_guide.md)\n- [CMD Service Guide](docs/zh/guide/cmd_service_guide.md)\n- [Client Guide](docs/zh/guide/client_guide.md)\n\n---\n\n## 🤝 Contributing\n\nContributions of all forms are welcome! For participation methods, refer to the [Contribution Guide](docs/zh/guide/contribution.md).\n\n---\n\n## 📄 License\n\nThis project is licensed under the [Apache 2.0](LICENSE) license.\n\n---\n\n## Star 历史\n\n[![Star History Chart](https://api.star-history.com/svg?repos=flowllm-ai/flowllm\u0026type=Date)](https://www.star-history.com/#flowllm-ai/flowllm\u0026Date)\n\n---\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/flowllm-ai/flowllm\"\u003eGitHub\u003c/a\u003e •\n  \u003ca href=\"https://flowllm-ai.github.io/flowllm/\"\u003eDocumentation\u003c/a\u003e •\n  \u003ca href=\"https://pypi.org/project/flowllm/\"\u003ePyPI\u003c/a\u003e\n\u003c/p\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowllm-ai%2Fflowllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflowllm-ai%2Fflowllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowllm-ai%2Fflowllm/lists"}