{"id":13703034,"url":"https://github.com/pragunbhutani/dbt-llm-agent","last_synced_at":"2026-01-19T13:37:06.129Z","repository":{"id":221956595,"uuid":"755860108","full_name":"pragunbhutani/dbt-llm-agent","owner":"pragunbhutani","description":"LLM based AI Agent to automate Data Analysis for dbt projects with remote MCP server","archived":false,"fork":false,"pushed_at":"2025-07-31T09:39:35.000Z","size":4318,"stargazers_count":158,"open_issues_count":4,"forks_count":28,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-01-08T17:27:55.431Z","etag":null,"topics":["agent","agentic-ai","ai","ai-data-analysis","data-analysis","data-analyst","dbt","llm","text-to-sql"],"latest_commit_sha":null,"homepage":"https://ragstar.vercel.app","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pragunbhutani.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-11T09:55:38.000Z","updated_at":"2026-01-05T22:10:50.000Z","dependencies_parsed_at":"2024-02-11T14:29:44.132Z","dependency_job_id":"3d0550fc-163d-4852-b062-ddb163aaf5c6","html_url":"https://github.com/pragunbhutani/dbt-llm-agent","commit_stats":null,"previous_names":["pragunbhutani/ragster","pragunbhutani/ragstar","pragunbhutani/dbt-llm-tools","pragunbhutani/dbt-llm-agent","pragunbhutani/dbt-analytics-agent"],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/pragunbhutani/dbt-llm-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pragunbhutani%2Fdbt-llm-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pragunbhutani%2Fdbt-llm-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pragunbhutani%2Fdbt-llm-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pragunbhutani%2Fdbt-llm-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pragunbhutani","download_url":"https://codeload.github.com/pragunbhutani/dbt-llm-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pragunbhutani%2Fdbt-llm-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28569418,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T12:50:50.164Z","status":"ssl_error","status_checked_at":"2026-01-19T12:50:42.704Z","response_time":67,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","agentic-ai","ai","ai-data-analysis","data-analysis","data-analyst","dbt","llm","text-to-sql"],"created_at":"2024-08-02T21:00:49.318Z","updated_at":"2026-01-19T13:37:06.116Z","avatar_url":"https://github.com/pragunbhutani.png","language":"Python","funding_links":[],"categories":["Utilities","📚 Projects (1974 total)","Building"],"sub_categories":["MCP Servers","Tools"],"readme":"# Ragstar — AI Data Analyst for dbt Projects\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/)\n[![Docker Compose](https://img.shields.io/badge/built%20with-Docker%20Compose-blue.svg)](https://docs.docker.com/compose/)\n\n\u003e **Ragstar is in public βeta.** Expect rapid changes and occasional rough edges.\n\n---\n\n## 1. What is Ragstar?\n\nRagstar connects to your **dbt** project, builds a knowledge base from models \u0026 documentation, and lets everyone ask data-related questions in plain English via a beautiful web dashboard or Slack. Under the hood Ragstar combines:\n\n- PostgreSQL + `pgvector` for fast similarity search\n- Embeddings + LLMs (OpenAI, Anthropic, etc.) for reasoning\n- A modern **Next.js** frontend \u0026 **Django** backend\n\n### Screenshots\n\n#### Dashboard\n\n![Dashboard](docs/dashboard.png)\n\n#### Integrations\n\n![Integrations](docs/integrations.png)\n\n#### Settings\n\n![Settings](docs/settings.png)\n\n---\n\n## 2. Quick start (🚀 2 commands)\n\n```bash\n# ① clone \u0026 prepare env file\n$ git clone https://github.com/pragunbhutani/ragstar.git \u0026\u0026 cd ragstar\n$ cp .env.example .env \u0026\u0026 ${EDITOR:-vi} .env  # ⇒ edit just the vars shown below\n\n# ② build \u0026 run everything\n$ docker compose up --build -d\n```\n\nWhen the containers are healthy:\n\n- Frontend: http://localhost:3000 (Next.js)\n- API: http://localhost:8000 (Django/DRF)\n- Flower: http://localhost:5555 (background tasks)\n\nRun first-time Django tasks:\n\n```bash\n# inside the running backend container\n$ docker compose exec backend-django \\\n    uv run python manage.py migrate\n```\n\n🎉 That's it — open http://localhost:3000, sign up for a new account and you're ready to start using Ragstar.\n\n---\n\n## 3. Environment variables\n\nOnly a handful of variables are truly **required** for a local/dev install. The rest are advanced overrides.\n\nRagstar keeps the default stack as lightweight as possible.  \nFor a **local `docker compose` run you only need 3 variables** – everything else has sane fall-backs.\n\n### 3.1 Required for local/dev\n\n| Var                   | Example                   | Purpose                                              |\n| --------------------- | ------------------------- | ---------------------------------------------------- |\n| `NEXTAUTH_SECRET`     | `openssl rand -base64 32` | Secret used by **next-auth** to sign session cookies |\n| `NEXTAUTH_URL`        | `http://localhost:3000`   | Public URL where the frontend is reachable           |\n| `NEXT_PUBLIC_API_URL` | `http://localhost:8000`   | Public URL of the Django API exposed to the browser  |\n\nCreate a `.env` file in the repo root and paste the three lines above (adjust URLs if you changed the ports).\n\n### 3.2 Common overrides (optional)\n\n| Var                                                                | Default                      | When you might set it                                                                                  |\n| ------------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------ |\n| `INTERNAL_API_URL`                                                 | `http://backend-django:8000` | Only needed when the frontend talks to the backend across Docker networks or remote hosts.             |\n| `ENVIRONMENT`                                                      | `local`                      | Switch between `local`, `development`, `production` behaviour inside Django settings.                  |\n| `APP_HOST`                                                         | —                            | Extra hostname to append to Django `ALLOWED_HOSTS` \u0026 CORS lists, e.g. your public Ngrok / Vercel host. |\n| `DATABASE_URL`                                                     | auto-generated by compose    | Point the backend \u0026 worker to your own Postgres instance.                                              |\n| `CELERY_BROKER_URL`                                                | `redis://redis:6379/0`       | Use an external Redis / RabbitMQ instead of the bundled one.                                           |\n| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION` | not set                      | **Only required when you disable LocalStack and want to store secrets in real AWS Parameter Store.**   |\n\n### 3.3 LLM Provider API Keys\n\nYou need to provide API keys for all the LLM providers you plan on using.\n\n```bash\nLLM_OPENAI_API_KEY=...\nLLM_ANTHROPIC_API_KEY=...\nLLM_GOOGLE_API_KEY=...\n```\n\n### 3.4 Postgres defaults (used by compose)\n\n```bash\nPOSTGRES_DB=ragstar\nPOSTGRES_USER=user\nPOSTGRES_PASSWORD=password\nPOSTGRES_PORT=5432\n```\n\nSet a single `EXTERNAL_POSTGRES_URL=\u003curl\u003e` to BYO Postgres (with the `pgvector` extension).\n\n---\n\n## 4. First-run onboarding\n\nAfter logging into the dashboard you'll be guided through these steps:\n\n1. **Add a dbt project** → _Projects › New_ (dbt **Cloud** recommended — just paste the service token). GitHub or local zip upload also supported.\n2. **Pick the dbt models you want to use for answering questions** → choose which model to use for ∙ questions ∙ embeddings ∙ SQL verification.\n3. **Configure Slack** (optional)\n   - Go to _Integrations › Slack_.\n   - Follow the inline manifest to create a Slack app.\n   - Paste **Bot Token**, **Signing Secret**, **App Token**.\n4. **Ask questions!** Use the chat on the dashboard or `/ask` in Slack.\n\n\u003e Other integrations (Metabase, Snowflake, MCP) are available under _Integrations_ but currently **βeta / experimental**. MCP server is temporarily disabled while we stabilise streaming support.\n\n---\n\n## 5 Managing the stack\n\nCommon operations are wrapped in one-liners:\n\n```bash\n# shell into backend or frontend\n$ docker compose exec backend-django bash\n$ docker compose exec frontend-nextjs sh\n\n# tail logs\n$ docker compose logs -f backend-django\n\n# stop / remove containers\ndocker compose down          # keep volumes\ndocker compose down -v       # destroy DB\n```\n\n---\n\n## 6. Local dev without Docker (advanced)\n\n1. Install **Python 3.10+**, **Node 18+**, **uv**, **pnpm**, and Postgres16+ with `pgvector`.\n2. `uv venv \u0026\u0026 source .venv/bin/activate \u0026\u0026 uv pip install -e backend_django/`\n3. `pnpm install --filter frontend_nextjs`\n4. Start services in two terminals:\n   - **Backend** — `cd backend_django \u0026\u0026 uv run python manage.py runserver 0.0.0.0:8000`\n   - **Frontend** — `cd frontend_nextjs \u0026\u0026 pnpm dev`\n5. Export the same env vars listed above.\n\nDocker is strongly recommended unless you're hacking on the codebase itself.\n\n---\n\n## 7. Contributing\n\nWe 💛 community PRs. Please file an issue first for major changes. Make sure `ruff`, `black`, `mypy`, and `eslint` pass before opening a pull request.\n\n---\n\n---\n\n## 8. MCP Server Integration (Self-Hosted Only)\n\nRagstar includes a **Model Context Protocol (MCP)** server that allows LLM clients like Claude.ai to directly access your dbt knowledge base. The MCP server provides secure, OAuth-authenticated access to your dbt models and project information.\n\n\u003e **⚠️ Important:** The MCP server can only be used with **self-hosted/open source** deployments of Ragstar. This is due to the **1:1 relationship** between an MCP client and server — each client needs its own dedicated server instance.\n\n### 8.1 What is the MCP Server?\n\nThe MCP server acts as a bridge between LLM clients (like Claude.ai) and your Ragstar knowledge base. It provides:\n\n- **Secure OAuth 2.0 authentication** with PKCE\n- **Organization-scoped access** to dbt models and projects\n- **Real-time data access** without manual imports\n- **Semantic search** capabilities across your dbt documentation\n- **Detailed model information** including SQL, lineage, and metadata\n\n### 8.2 Available MCP Tools\n\nThe MCP server exposes these tools to LLM clients:\n\n- `list_dbt_models` — Browse and filter dbt models by project, schema, or materialization\n- `search_dbt_models` — Semantic search for relevant models using natural language\n- `get_model_details` — Get detailed information about specific models including SQL and lineage\n- `get_project_summary` — Overview of connected dbt projects and their structure\n\n### 8.3 Setting Up the MCP Server\n\n#### Prerequisites\n\n1. **Self-hosted Ragstar** running with Docker Compose\n2. **dbt project connected** and models loaded in your knowledge base\n3. **LLM client** that supports MCP (e.g., Claude.ai, ChatGPT with MCP support)\n\n#### Configuration\n\nThe MCP server runs on port `8080` by default. Add these environment variables to your `.env` file:\n\n```bash\n# MCP Server Configuration\nMCP_AUTHORIZATION_BASE_URL=http://localhost:8000  # Your Django backend URL\nDJANGO_BACKEND_URL=http://localhost:8000          # Backend URL for MCP server\nALLOWED_ORIGINS=*                                 # Or specific origins for security\n```\n\n#### Starting the MCP Server\n\nThe MCP server is included in the Docker Compose stack:\n\n```bash\n# Start all services including MCP server\ndocker compose up -d\n\n# Check MCP server health\ncurl http://localhost:8080/health\n```\n\n### 8.4 OAuth 2.0 Flow Explanation\n\nThe MCP server implements a complete OAuth 2.0 authorization flow with organization-scoped access. Here's how it works:\n\n#### Flow Overview\n\n```mermaid\nsequenceDiagram\n    participant Claude as Claude.ai\n    participant MCP as MCP Server\n    participant Django as Django Backend\n    participant User as User Browser\n\n    Claude-\u003e\u003eMCP: 1. Request OAuth metadata\n    MCP-\u003e\u003eDjango: 2. Proxy metadata request\n    Django-\u003e\u003eMCP: 3. Return OAuth configuration\n    MCP-\u003e\u003eClaude: 4. OAuth server metadata\n\n    Claude-\u003e\u003eMCP: 5. Authorization request (with PKCE)\n    MCP-\u003e\u003eDjango: 6. Proxy authorization request\n    Django-\u003e\u003eUser: 7. Redirect to frontend login\n    User-\u003e\u003eDjango: 8. Authenticate via Next.js\n    Django-\u003e\u003eMCP: 9. Authorization code\n    MCP-\u003e\u003eClaude: 10. Return authorization code\n\n    Claude-\u003e\u003eMCP: 11. Exchange code for tokens\n    MCP-\u003e\u003eDjango: 12. Validate and exchange\n    Django-\u003e\u003eMCP: 13. Access \u0026 refresh tokens\n    MCP-\u003e\u003eClaude: 14. JWT tokens\n\n    Claude-\u003e\u003eMCP: 15. API calls with Bearer token\n    MCP-\u003e\u003eDjango: 16. Validate token \u0026 get user context\n    Django-\u003e\u003eMCP: 17. Organization-scoped data\n    MCP-\u003e\u003eClaude: 18. Formatted response\n```\n\n#### Key Components\n\n1. **OAuth Metadata Discovery**: MCP server exposes RFC 8414 compliant metadata endpoints\n2. **PKCE Security**: Uses Proof Key for Code Exchange for enhanced security\n3. **Auto-Registration**: Automatically registers new OAuth clients (like Claude.ai)\n4. **Organization Scoping**: All data access is automatically scoped to the user's organization\n5. **JWT Tokens**: Secure, stateless authentication using JSON Web Tokens\n\n#### Security Features\n\n- **PKCE (Proof Key for Code Exchange)**: Prevents authorization code interception attacks\n- **Organization Isolation**: Users can only access their organization's data\n- **Token Validation**: All API calls require valid JWT tokens\n- **Automatic Expiry**: Access tokens expire after 1 hour, refresh tokens after 7 days\n\n### 8.5 Connecting Claude.ai to Your MCP Server\n\n#### Step 1: Configure MCP in Claude.ai\n\n1. Go to Claude.ai settings\n2. Find the \"MCP Servers\" or \"Model Context Protocol\" section\n3. Add a new server with these details:\n   - **Server URL**: `http://localhost:8080` (or your public URL)\n   - **OAuth**: Enable OAuth 2.0 authentication\n   - **Auto-discovery**: Enable to automatically discover capabilities\n\n#### Step 2: Authorize the Connection\n\n1. Claude.ai will redirect you to your Ragstar login page\n2. Sign in with your Ragstar account\n3. You'll be redirected back to Claude.ai with authorization\n4. The connection will be linked to your organization\n\n#### Step 3: Start Using MCP Tools\n\nYou can now ask Claude.ai to:\n\n- \"List all my dbt models\"\n- \"Search for revenue-related models\"\n- \"Show me details about the customer_metrics model\"\n- \"What dbt projects do I have connected?\"\n\n### 8.6 Example MCP Interactions\n\n#### Listing Models\n\n```\nYou: \"What dbt models do I have available?\"\nClaude: [Calls list_dbt_models tool]\nClaude: \"You have 23 dbt models across 2 projects:\n- analytics_prod: 15 models (staging, marts schemas)\n- marketing_analytics: 8 models (staging, reporting schemas)\n...\"\n```\n\n#### Searching Models\n\n```\nYou: \"Find models related to customer revenue\"\nClaude: [Calls search_dbt_models with \"customer revenue\"]\nClaude: \"I found 3 relevant models:\n- customer_revenue_monthly (similarity: 0.95)\n- customer_ltv_calculation (similarity: 0.87)\n- revenue_attribution (similarity: 0.82)\n...\"\n```\n\n#### Getting Model Details\n\n```\nYou: \"Show me the SQL for customer_revenue_monthly\"\nClaude: [Calls get_model_details for \"customer_revenue_monthly\"]\nClaude: \"Here's the customer_revenue_monthly model:\n- Schema: marts\n- Materialization: table\n- SQL: SELECT customer_id, DATE_TRUNC('month', order_date) as month, SUM(amount) as revenue FROM...\"\n```\n\n### 8.7 Troubleshooting MCP Connection\n\n#### Common Issues\n\n1. **Authentication Fails**: Check that your Ragstar backend is accessible and you're logged in\n2. **No Models Found**: Ensure your dbt project is connected and models are loaded\n3. **Permission Errors**: Verify the user has access to the organization's dbt projects\n4. **Network Issues**: Check firewall settings and port accessibility\n\n#### Debug Commands\n\n```bash\n# Check MCP server health\ncurl http://localhost:8080/health\n\n# Test OAuth metadata\ncurl http://localhost:8080/.well-known/oauth-authorization-server\n\n# View MCP server logs\ndocker compose logs -f mcp-server\n```\n\n### 8.8 Production Deployment\n\nFor production use:\n\n1. **Use HTTPS**: Configure SSL certificates for secure connections\n2. **Restrict Origins**: Set specific allowed origins instead of `*`\n3. **Monitor Usage**: Track OAuth token usage and API calls\n4. **Scale Considerations**: Each client needs its own server instance\n\n### 8.9 Limitations\n\n- **1:1 Client-Server Relationship**: Each MCP client needs its own server instance\n- **Organization Scoping**: Users can only access their own organization's data\n- **Token Expiry**: Access tokens expire after 1 hour (refresh tokens after 7 days)\n- **Self-Hosted Only**: Not available in hosted/SaaS deployments\n\n---\n\n## 9. Contributing\n\nWe 💛 community PRs. Please file an issue first for major changes. Make sure `ruff`, `black`, `mypy`, and `eslint` pass before opening a pull request.\n\n---\n\n## 10. License\n\nRagstar is released under the MIT License — see [LICENSE](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpragunbhutani%2Fdbt-llm-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpragunbhutani%2Fdbt-llm-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpragunbhutani%2Fdbt-llm-agent/lists"}