{"id":28438439,"url":"https://github.com/shopware/knowledge-index","last_synced_at":"2025-06-28T17:31:53.021Z","repository":{"id":91792684,"uuid":"604688858","full_name":"shopware/knowledge-index","owner":"shopware","description":"A backend to manage document representations in vector stores and make them queryable","archived":false,"fork":false,"pushed_at":"2025-06-20T06:31:21.000Z","size":1442,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-20T07:39:54.651Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shopware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-21T15:32:07.000Z","updated_at":"2025-06-20T06:31:23.000Z","dependencies_parsed_at":"2023-12-08T10:30:43.226Z","dependency_job_id":"d9da8fc4-259f-49aa-9b08-5ac7b16bff74","html_url":"https://github.com/shopware/knowledge-index","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/shopware/knowledge-index","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shopware%2Fknowledge-index","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shopware%2Fknowledge-index/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shopware%2Fknowledge-index/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shopware%2Fknowledge-index/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shopware","download_url":"https://codeload.github.com/shopware/knowledge-index/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shopware%2Fknowledge-index/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262468765,"owners_count":23316182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-06T01:07:21.049Z","updated_at":"2025-06-28T17:31:53.012Z","avatar_url":"https://github.com/shopware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Setup\n\n\n## Docker\n\n**Environment variables**\n\n- `KNOWLEDGE_API_KEY` (required)\n- `OPENAI_API_KEY` (optional)\n- `POSTHOG_PROJECT_API_KEY` (optional)\n- `AZURE_OPENAI_API_KEY` (optional)\n- `AZURE_OPENAI_ENDPOINT` (optional)\n- `AZURE_OPENAI_DEPLOYMENT` (optional)\n- `AZURE_OPENAI_DEPLOYMENT_EMBEDDINGS` (optional)\n- `WEB_IMAGE` (optional), default: `ai-ml-web:latest` - you can skip building your local image and use pre-built\n  no-code image `bojanrajh/python:latest`\n\n**Examples**\n\n```bash\n$ export KNOWLEDGE_API_KEY=\"s0m3-r4nd0m-4p1-k3y-h34d3r-f0r-b4s1c-s3c00r1ty\"\n$ export OPENAI_API_KEY=\"sk-...\"\n$ export POSTHOG_PROJECT_API_KEY=\"phc_...\"\n$ export AZURE_OPENAI_API_KEY=\"...\"\n$ export AZURE_OPENAI_ENDPOINT=\"...\"\n$ export AZURE_OPENAI_DEPLOYMENT=\"...\"\n$ export AZURE_OPENAI_DEPLOYMENT_EMBEDDINGS=\"...\"\n$ export WEB_IMAGE=\"bojanrajh/python:latest\"\n```\n\nOr prefix `docker stack deploy` / `docker compose` commands with `env WEB_IMAGE=\"bojanrajh/python:latest\"`\n\nOptionally, build local docker image with:\n\n- `python` v3.10\n- `poetry` (+ install dependencies)\n- `uvicorn` server\n\n```bash\n$ docker build -t ai-ml-web:latest -f ./Dockerfile-web .\n# or for local version with tensorflow and pytest\n$ docker build -t ai-ml-web:latest -f ./Dockerfile-web-nocode .\n```\n\nInit docker swarm and deploy stack.\n\n```bash\n$ docker swarm init\n$ docker stack deploy ai-ml -c docker-compose.yml --prune\n```\n\nOr if you would like to use standard non-swarm docker compose:\n\n```bash\n$ docker compose up\n```\n\nOr if you would like to manually run a single container:\n\n```bash\n$ docker run -dit \\\n -p 10002:80 \\\n -v \"$PWD:/code\" \\\n -v \"$PWD/data:/data/docs\" \\\n -v \"$PWD/db:/data/db\" \\\n -v \"$PWD/cache:/data/cache\" \\\n ai-ml-web:latest\n```\n\nOpen your browser: [http://172.18.0.1:10002/](http://172.18.0.1:10002/)\nor [http://localhost:10002/](http://localhost:10002/)\n\nIf above links do not work, check IPs returned by the command:\n\n```bash\n$ docker container exec -it $(docker ps -f name=ai-ml_web --format \"{{.ID}}\") hostname -I\n```\n\nRun tests\n\n```bash\n$ docker container exec -it $(docker ps -f name=ai-ml_web --format \"{{.ID}}\") pytest\n```\n\n## Local\n\nInstall dependencies with poetry.\n\n```bash\n$ poetry install\n```\n\nEnter isolated poetry shell.\n\n```bash\n$ poetry shell\n```\n\nRun uvicorn web server.\n\n```bash\n$ uvicorn web.main:app --host 0.0.0.0 --port 80 --reload\n```\n\n## Devenv\n\nActivate direnv from your project root:\n\n```bash\n$ direnv allow\n```\n\nRun uvicorn web server.\n\n```bash\n$ uvicorn web.main:app --host 0.0.0.0 --port 80 --reload\n```\n\n# Endpoint examples\n\n## Upload input\n\nUpload .zip containing .md files.\n\n```bash\n$ curl \\\n -v \\\n -F content=@test.zip \\\n -F collection=test \\\n -H \"X-Shopware-Api-Key: your-api-key\" \\\n https://ai-ml.fly.dev/upload-input\n```\n\n## Ingestion\n\nIngest uploaded documents.\n\n```bash\n$ curl \\\n -X POST \\\n --data '{\"collection\":\"test\"}' \\\n -H \"Content-Type: application/json\" \\\n -H \"X-Shopware-Api-Key: your-api-key\" \\\n https://ai-ml.fly.dev/ingest\n```\n\n## Search\n\nSearch a collection.\n\n```bash\n$ curl \\\n -X POST \\\n --data '{\"search\":\"keywords\",\"collection\":\"test\"}' \\\n -H \"Content-Type: application/json\" \\\n https://ai-ml.fly.dev/query\n```\n\n## Neighbours\n```\n\nSearch a collection.\n\n```bash\n$ curl \\\n -X POST \\\n -H \"Content-Type: application/json\" \\\n --data '{\"query\":\"document/identifier/foo\",\"collection\":\"test\"}' \\\n https://ai-ml.fly.dev/neighbours\n```\n\n## Ask\n\nAsk AI engine to generate an answer to the question.\n\n```bash\n$ curl \\\n -X POST \\\n --data '{\"q\":\"What is Shopware?\",\"collection\":\"test\"}' \\\n https://ai-ml.fly.dev/question\n```\n\n## Remote debugging\n\n1. Install Chrome extension ModHeader or similar.\n2. Set `X-Shopware-Api-Key` header\n3. Download remote database (`.faiss` and `.pkl`) to your local computer - https://ai-ml.fly.dev/download/db/{collection}\n4. Extract it to your local `/data/db-{collection}` directory\n\n# Notes\n\nNotes:\n\n- auto-reload is supported with `--reload` parameter in the `uvicorn` entrypoint\n\nFly.io deployment:\n\n- See [./.github/workflows/test.yml](./.github/workflows/test.yml)\n- `fly auth docker --access-token ...`\n- `fly deploy -i ai-ml-server:latest` - push local image to fly.io, then deploy\n- `fly secrets set OPENAI_API_KEY=\"...\"` - or fallback to tensorflow\n- `fly secrets set KNOWLEDGE_API_KEY=\"...\"` - required\n- `fly secrets set POSTHOG_PROJECT_API_KEY=\"...\"` - optional\n- `fly secrets set AZURE_OPENAI_API_KEY=\"...\"` - optional\n- `fly secrets set AZURE_OPENAI_ENDPOINT=\"...\"` - optional\n- `fly secrets set AZURE_OPENAI_DEPLOYMENT=\"...\"` - optional\n- `fly secrets set AZURE_OPENAI_DEPLOYMENT_EMBEDDINGS=\"...\"` - optional\n- `fly volumes create data --region ams --size 1` + see [./fly-toml](./fly-toml)\n- `fly autoscale set min=2 max=4`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopware%2Fknowledge-index","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshopware%2Fknowledge-index","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopware%2Fknowledge-index/lists"}