{"id":34660123,"url":"https://github.com/linode/ai-quickstart-anythingllm","last_synced_at":"2026-05-27T10:32:43.596Z","repository":{"id":328602063,"uuid":"1111158855","full_name":"linode/ai-quickstart-anythingllm","owner":"linode","description":"One-click automated AI inference and RAG stack (vLLM + AnythingLLM + pgvector) on Akamai Cloud (Linode) RTX 4000 ada GPU instance.","archived":false,"fork":false,"pushed_at":"2025-12-15T07:37:59.000Z","size":124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-11T01:36:15.783Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-06T11:43:29.000Z","updated_at":"2025-12-15T07:38:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/linode/ai-quickstart-anythingllm","commit_stats":null,"previous_names":["linode/ai-quickstart-anythingllm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/linode/ai-quickstart-anythingllm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linode%2Fai-quickstart-anythingllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linode%2Fai-quickstart-anythingllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linode%2Fai-quickstart-anythingllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linode%2Fai-quickstart-anythingllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linode","download_url":"https://codeload.github.com/linode/ai-quickstart-anythingllm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linode%2Fai-quickstart-anythingllm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33562772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-27T02:00:06.184Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T18:52:01.115Z","updated_at":"2026-05-27T10:32:43.590Z","avatar_url":"https://github.com/linode.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Akamai Cloud - AI Quickstart : RAG Stack with AnythingLLM\n\nAutomated deployment script to run your private, self-hosted AI workspace on Akamai Cloud GPU instances. This stack combines vLLM for high-performance LLM inference with AnythingLLM - a full-stack application for building private AI assistants with RAG (Retrieval Augmented Generation), document chat, and agent capabilities.\n\n-----------------------------------------\n## 🚀 Quick Start\n\nJust run this single command:\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-anythingllm/main/deploy.sh | bash\n```\n\nThat's it! The script will download required files and guide you through the interactive deployment process.\n\n## ✨ Features\n- **Fully Automated Deployment**: Handles instance creation with real-time progress tracking\n- **Ready to use AI Stack**: vLLM for GPU-accelerated inference + AnythingLLM for enterprise AI workspace\n- **RAG \u0026 Document Chat**: Upload documents and chat with your data using AnythingLLM's built-in vector database\n- **AI Agents**: Build custom AI agents with tools and workflows\n- **Cross-Platform Support**: Works on macOS, Linux, and Windows (Git Bash/WSL)\n\n-----------------------------------------\n\n## 🏗️ What Gets Deployed\n\n\u003cimg src=\"docs/architecture.svg\" alt=\"Architecture\" align=\"left\" width=\"600\"/\u003e\n\n\u003cbr clear=\"left\"/\u003e\n\n### Linode GPU Instance with\n- Ubuntu 24.04 LTS with NVIDIA drivers\n- Docker \u0026 NVIDIA Container Toolkit\n- Systemd service for automatic startup on reboot\n\n### Docker Containers\n| | Service | Description |\n|:--:|:--|:--|\n| \u003cimg src=\"https://caddyserver.com/resources/images/favicon.png\" alt=\"Caddy\" width=\"32\"/\u003e | **Caddy** | Reverse proxy with automatic HTTPS (port 80/443) |\n| \u003cimg src=\"https://raw.githubusercontent.com/vllm-project/media-kit/main/vLLM-Logo.png\" alt=\"vLLM\" width=\"32\"/\u003e | **vLLM** | High-throughput LLM inference engine with OpenAI-compatible API (port 8000, internal) |\n| \u003cimg src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" alt=\"TEI\" width=\"32\"/\u003e | **Text Embeddings Inference** | GPU-accelerated embedding service using BAAI/bge-m3 model (port 8001, internal) |\n| \u003cimg src=\"https://wiki.postgresql.org/images/a/a4/PostgreSQL_logo.3colors.svg\" alt=\"pgvector\" width=\"32\"/\u003e | **pgvector** | PostgreSQL with vector similarity search extension for RAG storage (port 5432, internal) |\n| \u003cimg src=\"https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/png/anything-llm.png\" alt=\"AnythingLLM\" width=\"32\"/\u003e | **AnythingLLM** | Full-stack AI application with RAG, document chat, agents, and multi-user support (port 3001, internal) |\n\n### Models\n\n#### LLM: openai/gpt-oss-20b\n[gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) is OpenAI's first fully open-source LLM, released under Apache 2.0 license. Key characteristics:\n- **20B parameters**: Fits on a single RTX 4000 Ada GPU (20GB VRAM)\n- **High benchmark scores**: Competitive with larger models on reasoning and instruction-following tasks\n- **High throughput**: Optimized for fast token generation with vLLM inference engine\n\n#### Embedding: BAAI/bge-m3\n[BGE-M3](https://huggingface.co/BAAI/bge-m3) is a state-of-the-art multilingual embedding model. Key strengths:\n- **Multilingual**: Supports 100+ languages with strong cross-lingual retrieval\n- **Multi-functionality**: Supports dense, sparse (lexical), and multi-vector retrieval in one model\n- **Top performance**: Ranked #1 on MTEB multilingual leaderboard at release\n\n### What is AnythingLLM?\n[AnythingLLM](https://anythingllm.com/) is an open-source, full-stack application that turns any document, resource, or content into context for any LLM. Key features include:\n- **Document Intelligence**: Upload PDFs, Word docs, websites, and more - chat with your data instantly\n- **Vector Database**: Uses pgvector (PostgreSQL) for scalable, production-grade vector storage\n- **Multi-user Workspaces**: Create isolated workspaces for different projects or teams\n- **Privacy-First**: All data stays on your infrastructure - nothing leaves your server\n\n### RAG Pipeline Components\nThis deployment includes a complete RAG (Retrieval Augmented Generation) pipeline:\n- **Text Embeddings Inference**: Hugging Face's [TEI](https://github.com/huggingface/text-embeddings-inference) service running the [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) multilingual embedding model\n- **pgvector**: PostgreSQL extension for efficient vector similarity search, enabling fast document retrieval at scale\n\n-----------------------------------------\n\n## 📋 Requirements\n\n### Akamai Cloud Account\n- Active Linode account with GPU access enabled\n\n### Local System Requirements\n- **Required**: bash, curl, ssh, jq\n- **Note**: jq will be auto-installed if missing\n\n-----------------------------------------\n## 🚦 Getting Started\n\n### 1. Option A: Single Command Execution\n\nNo installation required - just run:\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-anythingllm/main/deploy.sh | bash\n```\n\n### 1. Option B: Download and Run\n\nDownload the script and run locally:\n\n```bash\ncurl -fsSLO https://raw.githubusercontent.com/linode/ai-quickstart-anythingllm/main/deploy.sh\nbash deploy.sh\n```\n\n### 1. Option C: Clone Repository\n\nIf you prefer to inspect or customize the scripts:\n\n```bash\ngit clone https://github.com/linode/ai-quickstart-anythingllm\ncd ai-quickstart-anythingllm\n./deploy.sh\n```\n\n\u003e [!NOTE]\n\u003e if you like to add more services check out docker compose template file\n\u003e ```\n\u003e vi /template/docker-compose.yml\n\u003e ```\n\u003e\n\n### 2. Follow Interactive Prompts\nThe script will ask you to:\n- Choose a region (e.g., us-east, eu-west)\n- Select GPU instance type\n- Provide instance label\n- Select or generate SSH keys\n- Confirm deployment\n\n### 3. Wait for Deployment\nThe script automatically:\n- Creates GPU instance in your Linode account\n- Monitors cloud-init installation progress\n- Waits for AnythingLLM health check\n- Waits for vLLM model loading\n\n### 4. Access Your Services\nOnce complete, you'll see:\n```\n🎉 Setup Complete!\n\n✅ Your AI LLM instance is now running!\n\n🌐 Access URLs:\n   AnythingLLM:  https://\u003cip-label\u003e.ip.linodeusercontent.com\n\n🔐 Access Credentials:\n   SSH:   ssh -i /path/to/your/key root@\u003cinstance-ip\u003e\n```\n\n### Configuration files in GPU Instance\n```\n   # Install script called by cloud-init service\n   /opt/ai-quickstart-anythingllm/install.sh\n\n   # docker compose file called by systemctl at startup\n   /opt/ai-quickstart-anythingllm/docker-compose.yml\n\n   # Caddy reverse proxy configuration\n   /opt/ai-quickstart-anythingllm/Caddyfile\n\n   # service definition\n   /etc/systemd/system/ai-quickstart-anythingllm.service\n```\n\n-----------------------------------------\n\n## 🗑️ Delete Instance\n\nTo delete a deployed instance:\n\n```bash\n# Remote execution\ncurl -fsSL https://raw.githubusercontent.com/linode/ai-quickstart-anythingllm/main/delete.sh | bash -s -- \u003cinstance_id\u003e\n\n# Or download script and run\ncurl -fsSLO https://raw.githubusercontent.com/linode/ai-quickstart-anythingllm/main/delete.sh\nbash delete.sh \u003cinstance_id\u003e\n```\n\nThe script will show instance details and ask for confirmation before deletion.\n\n-----------------------------------------\n\n## 📁 Project Structure\n\n```\nai-quickstart-anythingllm/\n├── deploy.sh                    # Main deployment script\n├── delete.sh                    # Instance deletion script\n└── template/\n    ├── cloud-init.yaml          # Cloud-init configuration\n    ├── docker-compose.yml       # Docker Compose configuration\n    ├── Caddyfile                # Caddy reverse proxy configuration\n    └── install.sh               # Post-boot installation script\n```\n\n-----------------------------------------\n## 🔒 Security\n\n**⚠️ IMPORTANT**: By default, ports 80 and 443 are exposed to the internet\n\n### Immediate Security Steps\n\n1. **Configure Cloud Firewall** (Recommended)\n   - Create Linode Cloud Firewall\n   - Restrict access to ports 80/443 by source IP\n   - Allow SSH (port 22) from trusted IPs only\n\n2. **SSH Security**\n   - SSH key authentication required\n   - Root password provided for emergency console access only\n\n-----------------------------------------\n## 🛠️ Useful Commands\n\n```bash\n# SSH into your instance\nssh -i /path/to/your/key root@\u003cinstance-ip\u003e\n\n# Check container status\ndocker ps -a\n\n# Check Docker containers log\ncd /opt/ai-quickstart-anythingllm \u0026\u0026 docker compose logs -f\n\n# Check systemd service status\nsystemctl status ai-quickstart-anythingllm.service\n\n# View systemd service logs\njournalctl -u ai-quickstart-anythingllm.service -n 100\n\n# Check cloud-init logs\ntail -f /var/log/cloud-init-output.log -n 100\n\n# Restart all services\nsystemctl restart ai-quickstart-anythingllm.service\n\n# Check NVIDIA GPU status\nnvidia-smi\n\n# Check vLLM loaded models\ncurl http://localhost:8000/v1/models\n\n# Check AnythingLLM health\ncurl http://localhost:3001/api/ping\n\n# Check container logs\ndocker logs vllm\ndocker logs anythingllm\ndocker logs embedding\ndocker logs pgvector\n\n# Check embedding service health\ncurl http://localhost:8001/health\n\n# Check pgvector status\ndocker exec pgvector pg_isready -U anythingllm\n```\n\n## 🤝 Contributing\n\nIssues and pull requests are welcome! For major changes, please open an issue first to discuss what you would like to change.\n\n## 📄 License\n\nThis project is licensed under the Apache License 2.0.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinode%2Fai-quickstart-anythingllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinode%2Fai-quickstart-anythingllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinode%2Fai-quickstart-anythingllm/lists"}