{"id":28190013,"url":"https://github.com/enso-labs/llm-server","last_synced_at":"2025-07-03T06:06:56.483Z","repository":{"id":205482399,"uuid":"714324803","full_name":"enso-labs/llm-server","owner":"enso-labs","description":"🤖 Open-source LLM server (OpenAI, Ollama, Groq, Anthropic) with support for HTTP, Streaming, Agents, RAG","archived":false,"fork":false,"pushed_at":"2025-06-10T22:08:19.000Z","size":29249,"stargazers_count":32,"open_issues_count":23,"forks_count":11,"subscribers_count":2,"default_branch":"development","last_synced_at":"2025-06-10T22:32:27.473Z","etag":null,"topics":["ai","anthropic","bedrock","fastapi","groq","langchain","large-language-models","minio","ollama","openai","python","redis","server","vercel"],"latest_commit_sha":null,"homepage":"https://promptengineersai.netlify.app","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/enso-labs.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-04T15:21:14.000Z","updated_at":"2025-06-10T22:08:24.000Z","dependencies_parsed_at":"2023-12-05T05:23:20.068Z","dependency_job_id":"1c354ccd-6f23-4d30-9539-7bac37961760","html_url":"https://github.com/enso-labs/llm-server","commit_stats":null,"previous_names":["promptengineers-ai/llm-server","enso-labs/llm-server"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/enso-labs/llm-server","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enso-labs%2Fllm-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enso-labs%2Fllm-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enso-labs%2Fllm-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enso-labs%2Fllm-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/enso-labs","download_url":"https://codeload.github.com/enso-labs/llm-server/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enso-labs%2Fllm-server/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263271501,"owners_count":23440396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","bedrock","fastapi","groq","langchain","large-language-models","minio","ollama","openai","python","redis","server","vercel"],"created_at":"2025-05-16T10:11:56.276Z","updated_at":"2025-07-03T06:06:56.469Z","avatar_url":"https://github.com/enso-labs.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚨 DEPRECATION NOTICE\n\n**⚠️ This repository is no longer maintained.**  \nAs of **June 10, 2025**, this project has been officially deprecated and archived. It will no longer receive updates, security patches, or support from the maintainers.\n\n### 👉 Recommended Action:\nWe recommend transitioning to [Ensō Orchestra](https://github.com/enso-labs/orchestra), which serves as the actively maintained and improved successor to this project.\n\n---\n\n\u003ch1 align=\"center\"\u003e\n  🤖 Prompt Engineers AI - LLM Server \n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\nFull LLM REST API with prompts, LLMs, Vector Databases, and Agents\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://promptengineers-ai.gitbook.io/documentation/open-source\"\u003e\u003cimg src=\"https://img.shields.io/badge/View%20Documentation-Docs-yellow\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://join.slack.com/t/promptengineersai/shared_invite/zt-21upjsftv-gX~gNjTCU~2HfbeM_ZwTEQ\"\u003e\u003cimg src=\"https://img.shields.io/badge/Join%20our%20community-Slack-blue\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/promptengineers-ai/llm-server/blob/master/docs/assets/llm-server.gif?raw=true\" width=\"600px\" /\u003e\n\u003c/p\u003e\n\n## 📖 Table of Contents\n\n- [Deploy](https://github.com/promptengineers-ai/llm-server/blob/development/docs/deploy)\n- [Tools](https://github.com/promptengineers-ai/llm-server/blob/development/docs/tools)\n\n## 🛠️ Setup Services\n```bash\n### Setup Docker Services\ndocker-compose up --build\n```\n\n## 🛠️ Setup Server\n\nBefore running the server make sure to take a look at `cp .example.env .env` see [Environment Variables](https://github.com/promptengineers-ai/llm-server?tab=readme-ov-file#environment-variables).\n\n```bash\n### Change into Backend directory\ncd backend\n\n### Setup Virtual Env\npython3 -m venv .venv\n\n### Activate Virtual Env\nsource .venv/bin/activate\n\n### Install Runtime \u0026 Dev Dependencies\npip install -r requirements.txt -r requirements-dev.txt -c constaints.txt\n\n### Install Runtime Dependencies\npip install -r requirements.txt -c constaints.txt\n\n### Migrate Database Schema\nalembic upgrade head\n\n### Seed Database Users\npython3 -m src.seeds.users 3\n\n### Run Application on local machine\nbash scripts/dev.sh\n```\n\n## 🛠️ Setup Client \n```bash\n### Change into Backend directory\ncd frontend\n\n### Install node_modules\nnpm install\n\n### Start Development Server\nnpm run dev\n```\n\n### Environment Variables\n\u003ctable border=\"1\" width=\"100%\"\u003e\n  \u003ctr\u003e\n    \u003cth\u003eVariable Name\u003c/th\u003e\n    \u003cth\u003eExample\u003c/th\u003e\n    \u003cth\u003eDescription\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_ENV\u003c/td\u003e\n    \u003ctd\u003e'development'\u003c/td\u003e\n    \u003ctd\u003eEnvironment where the application is running\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_VERSION\u003c/td\u003e\n    \u003ctd\u003e0.0.1\u003c/td\u003e\n    \u003ctd\u003eVersion of the application\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_SECRET\u003c/td\u003e\n    \u003ctd\u003ethis-is-top-secret\u003c/td\u003e\n    \u003ctd\u003eSecret key for the application\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_WORKERS\u003c/td\u003e\n    \u003ctd\u003e1\u003c/td\u003e\n    \u003ctd\u003eNumber of application workers\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_ADMIN_EMAIL\u003c/td\u003e\n    \u003ctd\u003eadmin@example.com\u003c/td\u003e\n    \u003ctd\u003eAdmin email for the application\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eAPP_ADMIN_PASS\u003c/td\u003e\n    \u003ctd\u003etest1234\u003c/td\u003e\n    \u003ctd\u003eAdmin password for the application\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTEST_USER_ID\u003c/td\u003e\n    \u003ctd\u003e0000000000000000000000000\u003c/td\u003e\n    \u003ctd\u003eTest user ID\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eDATABASE_URL\u003c/td\u003e\n    \u003ctd\u003emysql+aiomysql://admin:password@localhost:3306/llm_server\u003c/td\u003e\n    \u003ctd\u003eURL for the database\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ePINECONE_API_KEY\u003c/td\u003e\n    \u003ctd\u003e\u003c/td\u003e\n    \u003ctd\u003eAPI key for Pinecone services\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ePINECONE_ENV\u003c/td\u003e\n    \u003ctd\u003eus-east1-gcp\u003c/td\u003e\n    \u003ctd\u003ePinecone environment configuration\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ePINECONE_INDEX\u003c/td\u003e\n    \u003ctd\u003edefault\u003c/td\u003e\n    \u003ctd\u003eDefault Pinecone index used\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eREDIS_URL\u003c/td\u003e\n    \u003ctd\u003eredis://localhost:6379\u003c/td\u003e\n    \u003ctd\u003eURL for the Redis service\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eOPENAI_API_KEY\u003c/td\u003e\n    \u003ctd\u003esk-abc123...\u003c/td\u003e\n    \u003ctd\u003eDefault LLM OpenAI key\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eGROQ_API_KEY\u003c/td\u003e\n    \u003ctd\u003e\u003c/td\u003e\n    \u003ctd\u003eAPI key for accessing GROQ services\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eANTHROPIC_API_KEY\u003c/td\u003e\n    \u003ctd\u003e\u003c/td\u003e\n    \u003ctd\u003eAPI key for accessing Anthropic services\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eOLLAMA_BASE_URL\u003c/td\u003e\n    \u003ctd\u003ehttp://localhost:11434\u003c/td\u003e\n    \u003ctd\u003eBase URL for the Ollama service\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eSEARX_SEARCH_HOST_URL\u003c/td\u003e\n    \u003ctd\u003ehttp://localhost:8080\u003c/td\u003e\n    \u003ctd\u003eURL for the Searx search service\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eMINIO_HOST\u003c/td\u003e\n    \u003ctd\u003elocalhost:9000\u003c/td\u003e\n    \u003ctd\u003eURL to the Object storage\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eBUCKET\u003c/td\u003e\n    \u003ctd\u003emy-documents\u003c/td\u003e\n    \u003ctd\u003eName of Minio or S3 bucket\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eS3_REGION\u003c/td\u003e\n    \u003ctd\u003eus-east-1\u003c/td\u003e\n    \u003ctd\u003eRegion where the S3 bucket exists\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eACCESS_KEY_ID\u003c/td\u003e\n    \u003ctd\u003eAKIAIOSFODNN7EXAMPLE\u003c/td\u003e\n    \u003ctd\u003eIAM User Access Key ID \u0026#40;Optional\u0026#41;\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eACCESS_SECRET_KEY\u003c/td\u003e\n    \u003ctd\u003ewJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\u003c/td\u003e\n    \u003ctd\u003eSecret IAM Key \u0026#40;Optional\u0026#41;\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## 🚀 Roadmap\n\nHere are the upcoming features I'm (ryaneggleston@promptengineers.ai) is excited to bring to Prompt Engineers AI - LLM Server (More to come):\n\n- [x] 🤖 **Foundation Model Providers Supported (OpenAI, Anthropic, Ollama, Groq, Google... coming soon.)**\n- [x] 📸 **Multi-Modal Models Generation**\n- [x] 📑 **Retrieval Augmented Generation (RAG)**\n- [x] 🛠 **UI-Based Tool Configuration**\n- [x] 🖥 [**Code Interpreter**](https://github.com/promptengineers-ai/llm-server/blob/52b82eee1744d2b9543f788b835082c72fb8869c/backend/src/tools/__init__.py#L89) \n  - ⚠️ Use with Caution. Recommend [E2B Data Analysis](https://python.langchain.com/v0.2/docs/integrations/tools/e2b_data_analysis/)\n- [ ] 🌑 **Dark Mode**\n- [ ] 🎨 **Configure Custom Theme and Logos**    \n- [ ] 🤖 **Assistant Creation Capability**\n\nCreate an issue and lets start a discussion if you'd like to see a feature added to the roadmap.\n\n## 🤝 How to Contribute\n\nWe welcome contributions from the community, from beginners to seasoned developers. Here's how you can contribute:\n\n1. **Fork the repository**: Click on the 'Fork' button at the top right corner of the repository page on GitHub.\n\n2. **Clone the forked repository** to your local machine: `git clone \u003cforked_repo_link\u003e`.\n\n3. **Navigate to the project folder**: `cd llm-server`.\n\n4. **Create a new branch** for your changes: `git checkout -b \u003cbranch_name\u003e`.\n\n5. **Make your changes** in the new branch.\n\n6. **Commit your changes**: `git commit -am 'Add some feature'`.\n\n7. **Push to the branch**: `git push origin \u003cbranch_name\u003e`.\n\n8. **Open a Pull Request**: Go back to your forked repository on GitHub and click on 'Compare \u0026 pull request' to create a new pull request.\n\nPlease ensure that your code passes all the tests and if possible, add tests for new features. Always write a clear and concise commit message and pull request description.\n\n## 💡 Issues\n\nFeel free to submit issues and enhancement requests. We're always looking for feedback and suggestions.\n\n## 🤓 Maintainers\n\n- `Ryan Eggleston` - `ryaneggleston@promptengineers.ai`\n\n## 📜 License\n\nThis project is open-source, under the [MIT License](LICENSE). Feel free to use, modify, and distribute the code as you please.\n\nHappy Prompting! 🎉🎉\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenso-labs%2Fllm-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fenso-labs%2Fllm-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenso-labs%2Fllm-server/lists"}