{"id":49958744,"url":"https://github.com/wa91h/local-ai-toolkit","last_synced_at":"2026-05-18T01:09:56.581Z","repository":{"id":353946420,"uuid":"1166193790","full_name":"wa91h/local-ai-toolkit","owner":"wa91h","description":"A self-hosted AI toolkit running locally via Docker Compose, bundling an LLM gateway, workflow automation, and a chat UI — all backed by a shared PostgreSQL database.","archived":false,"fork":false,"pushed_at":"2026-05-15T11:02:40.000Z","size":80,"stargazers_count":16,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-15T13:07:50.340Z","etag":null,"topics":["ai","ai-agent","docker","docker-compose","litellm","llm","llm-gateway","llm-proxy","local-llm","n8n","ollama","openwebui","self-hosted","workflow"],"latest_commit_sha":null,"homepage":"","language":"Go Template","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wa91h.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-25T01:05:00.000Z","updated_at":"2026-05-15T11:02:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"bf87eb5f-eade-4f52-aa61-dc6b8207f701","html_url":"https://github.com/wa91h/local-ai-toolkit","commit_stats":null,"previous_names":["wa91h/local-ai-toolkit"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/wa91h/local-ai-toolkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wa91h%2Flocal-ai-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wa91h%2Flocal-ai-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wa91h%2Flocal-ai-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wa91h%2Flocal-ai-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wa91h","download_url":"https://codeload.github.com/wa91h/local-ai-toolkit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wa91h%2Flocal-ai-toolkit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33160168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-17T22:39:12.733Z","status":"ssl_error","status_checked_at":"2026-05-17T22:39:10.741Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-agent","docker","docker-compose","litellm","llm","llm-gateway","llm-proxy","local-llm","n8n","ollama","openwebui","self-hosted","workflow"],"created_at":"2026-05-18T01:09:55.861Z","updated_at":"2026-05-18T01:09:56.575Z","avatar_url":"https://github.com/wa91h.png","language":"Go Template","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Local Toolkit\n\nA self-hosted AI stack bundling an LLM gateway, workflow automation, and a chat UI — all backed by a shared PostgreSQL database. Deployable locally with Docker Compose or on any Kubernetes cluster via Helm.\n\nLiteLLM ships with no models — register the providers you want in the LiteLLM admin UI, then use them from Open WebUI.\n\n\u003e **⚠️ Warning:** This project is not production-ready out of the box. It has no TLS, no authentication proxy, no secrets management, and no backup strategy. It is safe for local use and trusted internal networks. See [Production Checklist](#production-checklist) before exposing it to the internet or running it with real data.\n\n## Services\n\n| Service        | Description                          | Docker Compose URL       |\n| -------------- | ------------------------------------ | ------------------------ |\n| **LiteLLM**    | OpenAI-compatible LLM API gateway    | http://localhost:4000    |\n| **LiteLLM UI** | Admin dashboard                      | http://localhost:4000/ui |\n| **n8n**        | Visual workflow automation           | http://localhost:5678    |\n| **Open WebUI** | ChatGPT-like interface               | http://localhost:3000    |\n\n## Architecture\n\n```\n┌─────────────┐              ┌─────────────┐\n│  Open WebUI │              │     n8n     │\n└──────┬──────┘              └──────┬──┬───┘\n       │                            │  │\n       │    OpenAI-compatible API   │  │ workflow storage\n       └─────────────┬──────────────┘  │\n                     ▼                 ▼\n              ┌────────────┐    ┌────────────┐\n              │  LiteLLM   │───▶│ PostgreSQL │\n              └────────────┘    └────────────┘\n```\n\nServices communicate internally by container/pod name.\n\n### Open WebUI ↔ LiteLLM\n\nOpen WebUI is wired to LiteLLM over its OpenAI-compatible API. The stack injects\n**only the host — never an API key**. On first login, open the LiteLLM UI at\n`http://localhost:4000/ui`, create a **virtual key**, and paste it into Open\nWebUI under **Settings → Connections**. The `LITELLM_MASTER_KEY` is\nintentionally never given to Open WebUI — use a scoped virtual key instead.\n\n## Prerequisites\n\n- **Docker Compose:** Docker Engine + Docker Compose plugin\n- **Kubernetes:** a running cluster + [Helm 3](https://helm.sh/docs/intro/install/)\n\n---\n\n## Deployment\n\n### Option A — Docker Compose (local)\n\n**1. Create the shared network** *(one-time)*\n```bash\ndocker network create ai-toolkit\n```\n\n**2. Configure environment**\n```bash\ncp .env.dist .env\n```\nEdit `.env`:\n```dotenv\nPOSTGRES_DB=postgres\nPOSTGRES_USER=postgres\nPOSTGRES_PASSWORD=\u003cyour_secure_password\u003e\n\nLITELLM_MASTER_KEY=sk-\u003cgenerate_a_key\u003e\nLITELLM_SALT_KEY=sk-\u003cgenerate_a_key\u003e\n\nTZ=America/New_York   # IANA timezone\n```\n\n**3. Start the stack**\n```bash\ndocker compose up -d\n```\n\n**4. Verify**\n```bash\ndocker compose ps   # wait until all services show \"healthy\"\n```\n\n**Disabling services** *(optional)*\n\nEdit `COMPOSE_PROFILES` in `.env` to remove services you don't need:\n```dotenv\n# All enabled (default)\nCOMPOSE_PROFILES=litellm,n8n,openwebui\n\n# Without n8n\nCOMPOSE_PROFILES=litellm,openwebui\n\n# LiteLLM only (no UI, no automation)\nCOMPOSE_PROFILES=litellm\n```\n\u003e Postgres always runs — it is not profile-gated since other services depend on it.\n\u003e Disabling `litellm` requires also removing `openwebui` (it depends on LiteLLM).\n\n---\n\n### Option B — Kubernetes / Helm\n\n**1. Install the chart**\n```bash\nhelm install ai-toolkit ./helm/ai-toolkit \\\n  --namespace ai-toolkit --create-namespace \\\n  --set litellm.masterKey=sk-... \\\n  --set litellm.saltKey=sk-... \\\n  --set postgresql.auth.password=...\n```\n\nFor repeatable installs, use a values file instead of `--set` flags:\n```bash\n# my-values.yaml\nlitellm:\n  masterKey: sk-...\n  saltKey: sk-...\n\npostgresql:\n  auth:\n    password: ...\n```\n```bash\nhelm install ai-toolkit ./helm/ai-toolkit \\\n  --namespace ai-toolkit --create-namespace \\\n  -f my-values.yaml\n```\n\n**2. Access services** *(without Ingress)*\n```bash\nkubectl port-forward svc/ai-toolkit-litellm 4000:4000\nkubectl port-forward svc/ai-toolkit-n8n 5678:5678\nkubectl port-forward svc/ai-toolkit-openwebui 3000:8080\n```\n\n**3. Enable Ingress** *(optional)*\n\nAdd to your values file:\n```yaml\ningress:\n  enabled: true\n  className: nginx\n  annotations:\n    cert-manager.io/cluster-issuer: letsencrypt-prod\n  hosts:\n    litellm: litellm.example.com\n    n8n: n8n.example.com\n    openwebui: chat.example.com\n  tls:\n    - secretName: ai-toolkit-tls\n      hosts:\n        - litellm.example.com\n        - n8n.example.com\n        - chat.example.com\n```\n\n**4. Use an external database** *(optional)*\n\nTo disable the bundled PostgreSQL and point to your own (e.g. RDS, Cloud SQL):\n```yaml\npostgresql:\n  enabled: false\n\nexternalDatabase:\n  host: my-db.example.com\n  port: 5432\n  username: postgres\n  password: ...\n  database: postgres\n```\nThe external server must already have `n8n` and `litellm` databases created.\n\n**5. Disable individual services** *(optional)*\n```yaml\nn8n:\n  enabled: false   # skip n8n if you don't need workflow automation\n\nopenwebui:\n  enabled: false   # skip Open WebUI if you use a different frontend\n```\n\n**Upgrade / uninstall**\n```bash\nhelm upgrade ai-toolkit ./helm/ai-toolkit -f my-values.yaml\nhelm uninstall ai-toolkit\n```\n\n---\n\n## Project Structure\n\n```\n.\n├── docker-compose.yml            # Docker Compose stack definition\n├── .env.dist                     # Environment variables template\n├── config/\n│   ├── litellm_config.yaml       # LiteLLM model registry (empty by default)\n│   └── init_db.sh                # Creates n8n \u0026 litellm databases on first run\n├── helm/\n│   └── ai-toolkit/               # Helm chart for Kubernetes\n│       ├── Chart.yaml\n│       ├── values.yaml           # All chart defaults, fully annotated\n│       └── templates/            # K8s manifests (Deployments, Services, PVCs, …)\n└── .github/\n    └── workflows/\n        └── validate.yml          # CI: validates compose file and Helm chart\n```\n\n## Models\n\nLiteLLM ships with **no models** pre-configured — register the providers you\nwant yourself. Any OpenAI-compatible or LiteLLM-supported provider works\n(OpenAI, Anthropic, Vertex, Groq, a local model server, …). See *Adding models\nto LiteLLM* below.\n\n---\n\n## Configuration\n\n### Adding models to LiteLLM\n\nLiteLLM ships with an empty model registry. Add models in\n`config/litellm_config.yaml` (Docker Compose) or via `litellm.extraModels`\nin your Helm values:\n\n```yaml\n- model_name: gpt-4o\n  litellm_params:\n    model: openai/gpt-4o\n    api_key: os.environ/OPENAI_API_KEY\n```\n\nPair any provider key with `litellm.extraEnv` (Helm) or the `litellm` service\nenvironment (Docker Compose). You can also add models at runtime in the\nLiteLLM UI — `STORE_MODEL_IN_DB` persists them to Postgres.\n\nDocker Compose: `docker compose restart litellm`\nKubernetes: `helm upgrade ai-toolkit ./helm/ai-toolkit -f my-values.yaml`\n\n### Calling the LiteLLM API directly\n\n```bash\ncurl http://localhost:4000/v1/models \\\n  -H \"Authorization: Bearer $LITELLM_MASTER_KEY\"\n```\n\n### PostgreSQL databases\n\nA single PostgreSQL 16 instance hosts two databases: `n8n` and `litellm`. The `config/init_db.sh` script creates them on first startup and is idempotent (safe to re-run).\n\nTo reset the database (Docker Compose):\n```bash\ndocker compose down\ndocker volume rm toolkit_postgres-data\ndocker compose up -d\n```\n\n---\n\n## Docker Compose — Useful Commands\n\n```bash\ndocker compose up -d                    # start all services\ndocker compose down                     # stop all services\ndocker compose ps                       # show service health\ndocker compose logs -f litellm          # tail logs for one service\ndocker compose restart litellm          # reload after config change\ndocker compose down -v \u0026\u0026 docker compose up -d  # full reset (deletes data)\n```\n\n---\n\n## Troubleshooting\n\n**`docker compose up` fails with \"network ai-toolkit not found\"**\n```bash\ndocker network create ai-toolkit\n```\n\n**Services stay `starting` or `unhealthy`**\nLiteLLM and n8n wait for Postgres to pass its healthcheck before starting. On first run, Postgres runs the init script which can take 20–30 s. Check progress:\n```bash\ndocker compose logs -f postgres\n```\n\n**Open WebUI shows no models**\nOpen WebUI ships with the LiteLLM *host* wired in but **no API key** — add a\nLiteLLM virtual key (from `http://localhost:4000/ui`) under **Settings →\nConnections**. LiteLLM itself serves no models until you register them (see\n*Adding models to LiteLLM* above):\n```bash\ndocker compose logs -f litellm\n```\n\n**n8n can't connect to the database**\nThe `POSTGRES_USER` and `POSTGRES_PASSWORD` in `.env` must match what was used when the `postgres-data` volume was first created. If they changed, reset the volume or revert the credentials.\n\n**Port already in use**\nChange the host-side port in `docker-compose.yml`, e.g. `\"4001:4000\"` for LiteLLM.\n\n**Helm pod stuck in `Pending`**\nThe cluster likely has no default StorageClass for PVCs. Either install one or set `persistence.storageClass` in your values to an available class:\n```bash\nkubectl get storageclass\n```\n\n---\n\n## Production Checklist\n\nThis stack ships with sane defaults for development and internal use. The items below are required before running it with real users or sensitive data.\n\n### Must-have\n\n| Item | How |\n|---|---|\n| **TLS** | Add cert-manager to your cluster and set `ingress.annotations` with a ClusterIssuer. All traffic is plain HTTP by default. |\n| **Authentication proxy** | Put [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/) or [Authelia](https://www.authelia.com/) in front of every public-facing service. None of the services (LiteLLM UI, n8n, Open WebUI) require login by default. |\n| **Secrets management** | `.env` files and K8s Secrets store credentials in plaintext / base64. Use [External Secrets Operator](https://external-secrets.io/) with AWS Secrets Manager, GCP Secret Manager, or Vault to inject secrets at runtime. |\n| **PostgreSQL backups** | Deploy a `pg_dump` CronJob or use [Velero](https://velero.io/) for volume snapshots. There is no backup strategy included. Data loss = permanent loss of n8n workflows and LiteLLM audit logs. |\n| **Pin image tags** | `main-stable`, `latest`, and `main` are mutable — a re-pull can silently introduce a breaking change. Pin to a specific version in `values.yaml` (e.g. `tag: \"v1.45.0\"`) for reproducible deployments. |\n\n### Should-have\n\n| Item | How |\n|---|---|\n| **n8n secure cookies** | `N8N_SECURE_COOKIE=false` is forced because there is no HTTPS by default. Once TLS is in place, remove that override from `n8n-deployment.yaml`. |\n| **Network policies** | By default every pod can reach every other pod. Add Kubernetes NetworkPolicy resources to restrict postgres and litellm to only their direct consumers. |\n| **Resource tuning** | The default resource limits in `values.yaml` are starting points. Profile your actual workload and adjust — especially PostgreSQL memory if you have large n8n execution history. |\n| **Monitoring** | Add [Prometheus](https://prometheus.io/) + [Grafana](https://grafana.com/) to the cluster. LiteLLM exposes Prometheus metrics at `/metrics`. |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwa91h%2Flocal-ai-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwa91h%2Flocal-ai-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwa91h%2Flocal-ai-toolkit/lists"}