{"id":13592798,"url":"https://github.com/msoedov/langcorn","last_synced_at":"2025-05-15T16:08:24.541Z","repository":{"id":153070805,"uuid":"627962120","full_name":"msoedov/langcorn","owner":"msoedov","description":"⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops","archived":false,"fork":false,"pushed_at":"2024-07-15T18:05:52.000Z","size":870,"stargazers_count":927,"open_issues_count":22,"forks_count":72,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-05-14T03:16:03.582Z","etag":null,"topics":["api","fastapi","langchain","langchain-python","large-language-models","llm","llmops","openai-api","rest-api","vercel","vercel-serverless-functions"],"latest_commit_sha":null,"homepage":"https://langcorn.vercel.app/docs","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/msoedov.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-14T15:28:45.000Z","updated_at":"2025-05-06T16:59:11.000Z","dependencies_parsed_at":"2024-05-03T22:28:13.677Z","dependency_job_id":"3ad5ceb6-1eec-4426-b680-de5889616aa0","html_url":"https://github.com/msoedov/langcorn","commit_stats":{"total_commits":95,"total_committers":4,"mean_commits":23.75,"dds":0.0736842105263158,"last_synced_commit":"f7fde47ee609c0623e7cc629eb01348bde3cf9c1"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msoedov%2Flangcorn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msoedov%2Flangcorn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msoedov%2Flangcorn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msoedov%2Flangcorn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/msoedov","download_url":"https://codeload.github.com/msoedov/langcorn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254374475,"owners_count":22060611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","fastapi","langchain","langchain-python","large-language-models","llm","llmops","openai-api","rest-api","vercel","vercel-serverless-functions"],"created_at":"2024-08-01T16:01:13.272Z","updated_at":"2025-05-15T16:08:24.516Z","avatar_url":"https://github.com/msoedov.png","language":"Python","funding_links":[],"categories":["Model Serving \u0026 Inference","Tools","A01_文本生成_文本对话","Python","工具","*Ops for AI"],"sub_categories":["Model Serving Frameworks","Services","大语言对话模型及数据","服务","Model Serving \u0026 Inference"],"readme":"# Langcorn\n\nLangCorn is an API server that enables you to serve LangChain models and pipelines with ease, leveraging the power of FastAPI for a robust and efficient experience.\n\n\u003cp\u003e\n\u003cimg alt=\"GitHub Contributors\" src=\"https://img.shields.io/github/contributors/msoedov/langcorn\" /\u003e\n\u003cimg alt=\"GitHub Last Commit\" src=\"https://img.shields.io/github/last-commit/msoedov/langcorn\" /\u003e\n\u003cimg alt=\"\" src=\"https://img.shields.io/github/repo-size/msoedov/langcorn\" /\u003e\n\u003cimg alt=\"Downloads\" src=\"https://static.pepy.tech/badge/langcorn\" /\u003e\n\u003cimg alt=\"GitHub Issues\" src=\"https://img.shields.io/github/issues/msoedov/langcorn\" /\u003e\n\u003cimg alt=\"GitHub Pull Requests\" src=\"https://img.shields.io/github/issues-pr/msoedov/langcorn\" /\u003e\n\u003cimg alt=\"Github License\" src=\"https://img.shields.io/github/license/msoedov/langcorn\" /\u003e\n\u003c/p\u003e\n\n## Features\n\n- Easy deployment of LangChain models and pipelines\n- Ready to use auth functionality\n- High-performance FastAPI framework for serving requests\n- Scalable and robust solution for language processing applications\n- Supports custom pipelines and processing\n- Well-documented RESTful API endpoints\n- Asynchronous processing for faster response times\n\n## 📦 Installation\n\nTo get started with LangCorn, simply install the package using pip:\n\n```shell\n\npip install langcorn\n```\n\n## ⛓️ Quick Start\n\nExample LLM chain ex1.py\n\n```python\n\nimport os\n\nfrom langchain import LLMMathChain, OpenAI\n\nos.environ[\"OPENAI_API_KEY\"] = os.environ.get(\"OPENAI_API_KEY\", \"sk-********\")\n\nllm = OpenAI(temperature=0)\nchain = LLMMathChain(llm=llm, verbose=True)\n```\n\nRun your LangCorn FastAPI server:\n\n```shell\nlangcorn server examples.ex1:chain\n\n\n[INFO] 2023-04-18 14:34:56.32 | api:create_service:75 | Creating service\n[INFO] 2023-04-18 14:34:57.51 | api:create_service:85 | lang_app='examples.ex1:chain':LLMChain(['product'])\n[INFO] 2023-04-18 14:34:57.51 | api:create_service:104 | Serving\n[INFO] 2023-04-18 14:34:57.51 | api:create_service:106 | Endpoint: /docs\n[INFO] 2023-04-18 14:34:57.51 | api:create_service:106 | Endpoint: /examples.ex1/run\nINFO:     Started server process [27843]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http://127.0.0.1:8718 (Press CTRL+C to quit)\n```\n\nor as an alternative\n\n```shell\npython -m langcorn server examples.ex1:chain\n\n```\n\nRun multiple chains\n\n```shell\npython -m langcorn server examples.ex1:chain examples.ex2:chain\n\n\n[INFO] 2023-04-18 14:35:21.11 | api:create_service:75 | Creating service\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:85 | lang_app='examples.ex1:chain':LLMChain(['product'])\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:85 | lang_app='examples.ex2:chain':SimpleSequentialChain(['input'])\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:104 | Serving\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /docs\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /examples.ex1/run\n[INFO] 2023-04-18 14:35:21.82 | api:create_service:106 | Endpoint: /examples.ex2/run\nINFO:     Started server process [27863]\nINFO:     Waiting for application startup.\nINFO:     Application startup complete.\nINFO:     Uvicorn running on http://127.0.0.1:8718 (Press CTRL+C to quit)\n```\n\nImport the necessary packages and create your FastAPI app:\n\n```python\n\nfrom fastapi import FastAPI\nfrom langcorn import create_service\n\napp:FastAPI = create_service(\"examples.ex1:chain\")\n```\n\nMultiple chains\n\n```python\n\nfrom fastapi import FastAPI\nfrom langcorn import create_service\n\napp:FastAPI = create_service(\"examples.ex2:chain\", \"examples.ex1:chain\")\n```\n\nor\n\n```python\nfrom fastapi import FastAPI\nfrom langcorn import create_service\n\napp: FastAPI = create_service(\n    \"examples.ex1:chain\",\n    \"examples.ex2:chain\",\n    \"examples.ex3:chain\",\n    \"examples.ex4:sequential_chain\",\n    \"examples.ex5:conversation\",\n    \"examples.ex6:conversation_with_summary\",\n    \"examples.ex7_agent:agent\",\n)\n\n```\n\nRun your LangCorn FastAPI server:\n\n```shell\n\nuvicorn main:app --host 0.0.0.0 --port 8000\n```\n\nNow, your LangChain models and pipelines are accessible via the LangCorn API server.\n\n## Docs\n\nAutomatically served FastAPI doc\n[Live example](https://langcorn-ift9ub8zg-msoedov.vercel.app/docs#/) hosted on vercel.\n\n![](https://res.cloudinary.com/dq0w2rtm9/image/upload/c_pad,b_auto:predominant,fl_preserve_transparency/v1681817836/Screen_Shot_2023-04-18_at_14.36.00_ms2thb.jpg?_s=public-apps)\n\n## Auth\n\nIt possible to add a static api token auth by specifying `auth_token`\n\n```shell\npython langcorn server examples.ex1:chain examples.ex2:chain --auth_token=api-secret-value\n```\n\nor\n\n```python\napp:FastAPI = create_service(\"examples.ex1:chain\", auth_token=\"api-secret-value\")\n```\n\n## Custom API KEYs\n\n```shell\nPOST http://0.0.0.0:3000/examples.ex6/run\nX-LLM-API-KEY: sk-******\nContent-Type: application/json\n```\n\n## Handling memory\n\n```json\n{\n  \"history\": \"string\",\n  \"input\": \"What is brain?\",\n  \"memory\": [\n    {\n      \"type\": \"human\",\n      \"data\": {\n        \"content\": \"What is memory?\",\n        \"additional_kwargs\": {}\n      }\n    },\n    {\n      \"type\": \"ai\",\n      \"data\": {\n        \"content\": \" Memory is the ability of the brain to store, retain, and recall information. It is the capacity to remember past experiences, facts, and events. It is also the ability to learn and remember new information.\",\n        \"additional_kwargs\": {}\n      }\n    }\n  ]\n}\n\n```\n\nResponse:\n\n```json\n{\n  \"output\": \" The brain is an organ in the human body that is responsible for controlling thought, memory, emotion, and behavior. It is composed of billions of neurons that communicate with each other through electrical and chemical signals. It is the most complex organ in the body and is responsible for all of our conscious and unconscious actions.\",\n  \"error\": \"\",\n  \"memory\": [\n    {\n      \"type\": \"human\",\n      \"data\": {\n        \"content\": \"What is memory?\",\n        \"additional_kwargs\": {}\n      }\n    },\n    {\n      \"type\": \"ai\",\n      \"data\": {\n        \"content\": \" Memory is the ability of the brain to store, retain, and recall information. It is the capacity to remember past experiences, facts, and events. It is also the ability to learn and remember new information.\",\n        \"additional_kwargs\": {}\n      }\n    },\n    {\n      \"type\": \"human\",\n      \"data\": {\n        \"content\": \"What is brain?\",\n        \"additional_kwargs\": {}\n      }\n    },\n    {\n      \"type\": \"ai\",\n      \"data\": {\n        \"content\": \" The brain is an organ in the human body that is responsible for controlling thought, memory, emotion, and behavior. It is composed of billions of neurons that communicate with each other through electrical and chemical signals. It is the most complex organ in the body and is responsible for all of our conscious and unconscious actions.\",\n        \"additional_kwargs\": {}\n      }\n    }\n  ]\n}\n```\n\n## LLM kwargs\n\nTo override the default LLM params per request\n\n```shell\nPOST http://0.0.0.0:3000/examples.ex1/run\nX-LLM-API-KEY: sk-******\nX-LLM-TEMPERATURE: 0.7\nX-MAX-TOKENS: 256\nX-MODEL-NAME: gpt5\nContent-Type: application/json\n```\n\n## Custom run function\n\nSee ex12.py\n\n```python\n\nchain = LLMChain(llm=llm, prompt=prompt, verbose=True)\n\n\n# Run the chain only specifying the input variable.\n\n\ndef run(query: str) -\u003e Joke:\n    output = chain.run(query)\n    return parser.parse(output)\n\napp: FastAPI = create_service(\"examples.ex12:run\")\n\n```\n\n## Documentation\n\nFor more detailed information on how to use LangCorn, including advanced features and customization options, please refer to the official documentation.\n\n## 👋 Contributing\n\nContributions to LangCorn are welcome! If you'd like to contribute, please follow these steps:\n\n- Fork the repository on GitHub\n- Create a new branch for your changes\n- Commit your changes to the new branch\n- Push your changes to the forked repository\n- Open a pull request to the main LangCorn repository\n\nBefore contributing, please read the contributing guidelines.\n\n## License\n\nLangCorn is released under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsoedov%2Flangcorn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmsoedov%2Flangcorn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsoedov%2Flangcorn/lists"}