{"id":13593796,"url":"https://github.com/stoyan-stoyanov/llmflows","last_synced_at":"2025-05-14T20:05:12.696Z","repository":{"id":183253046,"uuid":"658523581","full_name":"stoyan-stoyanov/llmflows","owner":"stoyan-stoyanov","description":"LLMFlows - Simple, Explicit and Transparent LLM Apps","archived":false,"fork":false,"pushed_at":"2025-02-20T16:53:45.000Z","size":37859,"stargazers_count":691,"open_issues_count":18,"forks_count":34,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-02T09:42:50.801Z","etag":null,"topics":["ai","chatgpt","gpt-4","llm","llm-inference","llmops","llms","machine-learning","openai","prompt-engineering","python","question-answering","vector-database"],"latest_commit_sha":null,"homepage":"https://llmflows.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stoyan-stoyanov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-26T01:24:15.000Z","updated_at":"2025-03-31T05:59:34.000Z","dependencies_parsed_at":"2023-09-25T02:58:03.864Z","dependency_job_id":"357bacba-c58b-416d-9048-4b0909452fae","html_url":"https://github.com/stoyan-stoyanov/llmflows","commit_stats":null,"previous_names":["stoyan-stoyanov/llmflows"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stoyan-stoyanov%2Fllmflows","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stoyan-stoyanov%2Fllmflows/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stoyan-stoyanov%2Fllmflows/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stoyan-stoyanov%2Fllmflows/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stoyan-stoyanov","download_url":"https://codeload.github.com/stoyan-stoyanov/llmflows/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248027406,"owners_count":21035594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chatgpt","gpt-4","llm","llm-inference","llmops","llms","machine-learning","openai","prompt-engineering","python","question-answering","vector-database"],"created_at":"2024-08-01T16:01:24.708Z","updated_at":"2025-04-09T11:04:02.288Z","avatar_url":"https://github.com/stoyan-stoyanov.png","language":"Python","funding_links":[],"categories":["*Ops for AI","Python","LLMOps","SDK, Libraries, Frameworks","其他LLM框架","Other LLM Frameworks"],"sub_categories":["AI Orchestration \u0026 Deployment","Observability","Popular-LLM","Python","文章","Videos Playlists"],"readme":"\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/stoyan-stoyanov/llmflows/blob/main/docs/logo_with_subheading.png\"/\u003e\n\u003c/p\u003e\n\n\n[![Twitter](https://img.shields.io/twitter/follow/LLMFlows?style=social)](https://twitter.com/LLMFlows)\n![Pylint workflow](https://github.com/stoyan-stoyanov/llmflows/actions/workflows/pylint.yml/badge.svg)\n![License](https://img.shields.io/github/license/stoyan-stoyanov/llmflows)\n![PyPi](https://img.shields.io/pypi/v/llmflows)\n![Stars](https://img.shields.io/github/stars/stoyan-stoyanov/llmflows?style=social)\n![Release date](https://img.shields.io/github/release-date/stoyan-stoyanov/llmflows?style=social)\n\nDocumentation: \u003ca href=\"https://llmflows.readthedocs.io/\" target=\"_blank\"\u003ehttps://llmflows.readthedocs.io\u003c/a\u003e\u003c/br\u003e\nPyPI: \u003ca href=\"https://pypi.org/project/llmflows/\" target=\"_blank\"\u003ehttps://pypi.org/project/llmflows\u003c/a\u003e\u003c/br\u003e\nTwitter: \u003ca href=\"https://twitter.com/LLMFlows/\" target=\"_blank\"\u003ehttps://twitter.com/LLMFlows\u003c/a\u003e\u003c/br\u003e\nSubstack: \u003ca href=\"https://llmflows.substack.com/\" target=\"_blank\"\u003ehttps://llmflows.substack.com\u003c/a\u003e\u003c/br\u003e\n\n## 🤖 About LLM Flows\nLLMFlows is a framework for building simple, explicit, and transparent LLM(Large Language Model) applications such as chatbots, question-answering systems, and agents.\n\nAt its core, LLMFlows provides a minimalistic set of abstractions that allow you to utilize LLMs and vector stores and build well-structured and explicit apps that don't have hidden prompts or LLM calls. LLM Flows ensures complete transparency for each component, making monitoring, maintenance, and debugging easy. \n\n\n## 📦 Installation\n```\npip install llmflows\n```\n\n## 🧭 Philosophy\n\n### **Simple**\nOur goal is to build a simple, well-documented framework with minimal abstractions that \nallow users to build flexible LLM-powered apps without compromising on capabilities.\n\n### **Explicit**\nWe want to create an explicit API enabling users to write clean and readable code while \neasily creating complex flows of LLMs interacting with each other. LLMFlows' classes \ngive users full control and do not have any hidden prompts or LLM calls. \n\n### **Transparent**\nWe aim to help users have full transparency on their LLM-powered apps by providing \ntraceable flows and complete information for each app component, making it easy to \nmonitor, maintain, and debug.\n\n## ▶️ Live Demo\nCheck out [LLM-99](https://llm-99.vercel.app/) - a demo app that uses LLMs to explain \nsuperconductors in simple terms. The app is built with LLMFlows, and FastAPI and uses \nPinecone to store document embeddings created from Wikipedia articles. You can find \nthe source code for this demo app and other examples in our \n[examples folder](https://github.com/stoyan-stoyanov/llmflows/tree/main/examples/llmflows_in_fastapi/superconductor).\n\n![LLM-99](https://github.com/stoyan-stoyanov/llmflows/blob/main/docs/llm99_2_small.gif)\n\n## 🧪 Getting Started\n### LLMs\nLLMs are one of the main abstractions in LLMFlows. LLM classes are wrappers around LLM \nAPIs such as OpenAI's APIs. They provide methods for configuring and calling these APIs, \nretrying failed calls, and formatting the responses.\n\n```python\nfrom llmflows.llms import OpenAI\n\nllm = OpenAI(api_key=\"\u003cyour-openai-api-key\u003e\")\n\nresult, call_data, model_config = llm.generate(\n   prompt=\"Generate a cool title for an 80s rock song\"\n)\n```\n\n\n### PromptTemplates\nThe `PromptTemplate` class allows us to create strings with variables that we can fill \nin dynamically later on. Once a prompt template object is created an actual prompt can \nbe generated by providing the required variables.\n\n```python\nfrom llmflows.llms import OpenAI\nfrom llmflows.prompts import PromptTemplate\n\n\nprompt_template = PromptTemplate(\n    prompt=\"Generate a title for a 90s hip-hop song about {topic}.\"\n)\nllm_prompt = prompt_template.get_prompt(topic=\"friendship\")\n\nprint(llm_prompt)\n\nllm = OpenAI(api_key=\"\u003cyour-openai-api-key\u003e\")\nsong_title = llm.generate(llm_prompt)\n\nprint(song_title)\n```\n\n### Chat LLMs\nUnlike regular LLMs that only require a prompt to generate text, chat LLMs require a \nconversation history. The conversation history is represented \nas a list of messages between a user and an assistant. This conversation history is \nsent to the model, and a new message is generated based on it.\n\nLLMFlows provides a `MessageHistory` class to manage the required conversation history \nfor chat LLMs.\n\nYou can build a simple chatbot by using the `OpenAIChat` and `MessageHistory` classes:\n\n```python\nfrom llmflows.llms import OpenAIChat, MessageHistory\n\nllm = OpenAIChat(api_key=\"\u003cyour-openai-api-key\u003e\")\nmessage_history = MessageHistory()\n\nwhile True:\n    user_message = input(\"You:\")\n    message_history.add_user_message(user_message)\n\n    llm_response, call_data, model_config = llm.generate(message_history)\n    message_history.add_ai_message(llm_response)\n\n    print(f\"LLM: {llm_response}\")\n```\n\n### LLM Flows\nOften times, real-world applications can be more complex and can have dependencies \nbetween prompts and LLM calls. For example:\n\n![Complex flow](https://github.com/stoyan-stoyanov/llmflows/blob/main/docs/complex_flow.png)\n\nWhen you want to build apps with complex dependencies you can use the `Flow` and \n`Flowstep` classes. LLMFlows will figure out the dependencies and make sure each \nflowstep runs only when all its dependencies are met:\n\n```python\nfrom llmflows.flows import Flow, FlowStep\nfrom llmflows.llms import OpenAI\nfrom llmflows.prompts import PromptTemplate\n\nopenai_llm = OpenAI(api_key=\"\u003cyour-openai-api-key\u003e\")\n\n# Create prompt templates\ntitle_template = PromptTemplate(\"What is a good title of a movie about {topic}?\")\nsong_template = PromptTemplate(\n    \"What is a good song title of a soundtrack for a movie called {movie_title}?\"\n)\ncharacters_template = PromptTemplate(\n    \"What are two main characters for a movie called {movie_title}?\"\n)\nlyrics_template = PromptTemplate(\n    \"Write lyrics of a movie song called {song_title}. The main characters are \"\n    \"{main_characters}\"\n)\n\n# Create flowsteps\nmovie_title_flowstep = FlowStep(\n    name=\"Movie Title Flowstep\",\n    llm=openai_llm,\n    prompt_template=title_template,\n    output_key=\"movie_title\",\n)\n\nsong_title_flowstep = FlowStep(\n    name=\"Song Title Flowstep\",\n    llm=openai_llm,\n    prompt_template=song_template,\n    output_key=\"song_title\",\n)\n\ncharacters_flowstep = FlowStep(\n    name=\"Characters Flowstep\",\n    llm=openai_llm,\n    prompt_template=characters_template,\n    output_key=\"main_characters\",\n)\n\nsong_lyrics_flowstep = FlowStep(\n    name=\"Song Lyrics Flowstep\",\n    llm=openai_llm,\n    prompt_template=lyrics_template,\n    output_key=\"song_lyrics\",\n)\n\n# Connect flowsteps\nmovie_title_flowstep.connect(song_title_flowstep, characters_flowstep, song_lyrics_flowstep)\nsong_title_flowstep.connect(song_lyrics_flowstep)\ncharacters_flowstep.connect(song_lyrics_flowstep)\n\n# Create and run Flow\nsoundtrack_flow = Flow(movie_title_flowstep)\nresults = soundtrack_flow.start(topic=\"friendship\", verbose=True)\n```\n\n### Async Flows\nSometimes multiple flow steps can run in parallel if all their dependencies are met. \nFor cases like this, LLMFlows provides async classes to improve the runtime of any \ncomplex flow by running flow steps that already have all their required inputs in \nparallel.\n\n```python\n\n...\n\nmovie_title_flowstep = AsyncFlowStep(\n    name=\"Movie Title Flowstep\",\n    llm=openai_llm,\n    prompt_template=title_template,\n    output_key=\"movie_title\",\n)\n\nsong_title_flowstep = FlowStep(\n    name=\"Song Title Flowstep\",\n    llm=openai_llm,\n    prompt_template=song_template,\n    output_key=\"song_title\",\n)\n\ncharacters_flowstep = AsyncFlowStep(\n    name=\"Characters Flowstep\",\n    llm=openai_llm,\n    prompt_template=characters_template,\n    output_key=\"main_characters\",\n)\n\nsong_lyrics_flowstep = AsyncFlowStep(\n    name=\"Song Lyrics Flowstep\",\n    llm=openai_llm,\n    prompt_template=lyrics_template,\n    output_key=\"song_lyrics\",\n)\n\n...\n\n```\n\nFor more examples, such as integrating vector databases, creating question-answering \napps, agents, and web applications visit our \n[documentation](https://llmflows.readthedocs.io/en/latest/) or check out the \n[examples folder](https://github.com/stoyan-stoyanov/llmflows/tree/main/examples) of the repository.\n\n## 🛠️ Features\n\n### **LLMs**\n- Utilize LLMs such as OpenAI's ChatGPT to generate natural language text.\n- Configure LLM classes easily, choosing specific models, parameters, and settings.\n- Benefit from automatic retries when model calls fail, ensuring reliable LLM \n  interactions.\n\n### **Prompt Templates**\n- Create dynamic prompts using Prompt Templates, providing flexible and customizable \n  text generation.\n- Define variables within prompts to generate prompt strings tailored to specific \n  inputs.\n\n### **Flows and FlowSteps**\n- Structure LLM applications using Flows and FlowSteps, providing a clear and organized framework for executing LLM interactions.\n- Connect flow steps to pass outputs as inputs, facilitating seamless data flow and\n    maintaining a transparent LLM pipeline.\n- Leverage Async Flows to run LLMs in parallel when all their inputs are available, \n  optimizing performance and efficiency.\n- Incorporate custom string manipulation functions directly into flows, allowing \n  specialized text transformations without relying solely on LLM calls.\n\n### **VectorStore Integrations**\n- Integrate with vector databases like Pinecone using the VectorStoreFlowStep, \n  empowering efficient and scalable storage and retrieval of vector embeddings.\n- Leverage vector databases for seamless storage and querying of vectors, enabling straightforward integration with LLM-powered applications.\n\n### **Callbacks**\n- Execute callback functions at different stages within flow steps, enabling enhanced customization, logging, tracing, or other specific integrations.\n- Utilize callbacks to comprehensively control and monitor LLM-powered apps, ensuring \n  clear visibility into the execution process.\n\n### **Explicit API and Full Transparency**\nWith LLMFlows you have the full control to create explicit applications without any hidden prompts or predefined behaviors.\n\nIn addition LLMFlows allows you to answer questions such as:\n\n- When was a particular flowstep run?\n- How much time did it take?\n- What were the input variables?\n- What was the prompt template?\n- What did the prompt look like?\n- What was the exact configuration of the model?\n- How many times did we retry the request?\n- What was the raw data the API returned?\n- How many tokens were used?\n- What was the final result?\n\n## 📚 User Guide\n[Introduction](https://llmflows.readthedocs.io/en/latest/user_guide/Introduction/)\u003cbr/\u003e\n[LLM Flows](https://llmflows.readthedocs.io/en/latest/user_guide/LLM%20Flows/)\u003cbr/\u003e\n[Async Flows](https://llmflows.readthedocs.io/en/latest/user_guide/Async%20Flows/)\u003cbr/\u003e\n[Vector Stores](https://llmflows.readthedocs.io/en/latest/user_guide/Vector%20Stores/)\u003cbr/\u003e\n[Question Answering](https://llmflows.readthedocs.io/en/latest/user_guide/Question%20Answering/)\u003cbr/\u003e\n[Vector Stores in Flows](https://llmflows.readthedocs.io/en/latest/user_guide/Vector%20Stores%20in%20Flows/)\u003cbr/\u003e\n[Functional FlowSteps](https://llmflows.readthedocs.io/en/latest/user_guide/Functional%20FlowSteps/)\u003cbr/\u003e\n[Callbacks](https://llmflows.readthedocs.io/en/latest/user_guide/Callbacks/)\u003cbr/\u003e\n[LLM Flows with FastAPI](https://llmflows.readthedocs.io/en/latest/user_guide/LLMFlows%20with%20FastAPI/)\u003cbr/\u003e\n[Agents](https://llmflows.readthedocs.io/en/latest/user_guide/Agents/)\u003cbr/\u003e\n\n## ▶️ Examples\nYou can find all examples used in the User Guide in the [examples folder](https://github.com/stoyan-stoyanov/llmflows/tree/main/examples) of the repository.\n\n## 📃 License\nLLMFlows is covered by the MIT license. For more information, check [`LICENCE`](https://github.com/stoyan-stoyanov/llmflows/blob/main/LICENSE).\n\n## ❤️ How you can help?\nThank you for spending time going over our README! \n\nIf you like the project please consider giving it a star, sharing it with friends or on social media.\n\nIf you've tried LLMFlows and have some issues, feedback or ideas feel free to open an issue or reach out!\n\nIf you find LLMFlows exciting and you are considering contributing, please check [`CONTRIBUTING.md`](https://github.com/stoyan-stoyanov/llmflows/blob/main/CONTRIBUTING.md).\n\n## ✉️ Contact\nIf you want to reach out please don't hesitate to connect on the following social media:\n\n[Threads](https://www.threads.net/@sptstoyanov)\u003cbr/\u003e\n[LinkedIn](https://www.linkedin.com/in/spstoyanov/)\u003cbr/\u003e\n[Twitter](https://twitter.com/stoyanpstoyanov)\u003cbr/\u003e\n\nI would love to hear from you!\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstoyan-stoyanov%2Fllmflows","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstoyan-stoyanov%2Fllmflows","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstoyan-stoyanov%2Fllmflows/lists"}