{"id":20122894,"url":"https://github.com/thm-graphs/genai-stack","last_synced_at":"2025-10-07T04:21:51.626Z","repository":{"id":243069728,"uuid":"811300520","full_name":"THM-Graphs/genai-stack","owner":"THM-Graphs","description":"Fork of neo4j's genai stack adopted to Sozinianer dataset","archived":false,"fork":false,"pushed_at":"2024-12-13T08:20:07.000Z","size":2487,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T19:51:10.681Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/THM-Graphs.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-06T10:31:10.000Z","updated_at":"2024-12-13T08:20:11.000Z","dependencies_parsed_at":"2024-06-06T15:24:43.575Z","dependency_job_id":"88b0db6d-f2b7-4326-b93d-4800083e61ac","html_url":"https://github.com/THM-Graphs/genai-stack","commit_stats":null,"previous_names":["thm-graphs/genai-stack"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/THM-Graphs/genai-stack","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THM-Graphs%2Fgenai-stack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THM-Graphs%2Fgenai-stack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THM-Graphs%2Fgenai-stack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THM-Graphs%2Fgenai-stack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/THM-Graphs","download_url":"https://codeload.github.com/THM-Graphs/genai-stack/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/THM-Graphs%2Fgenai-stack/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273728075,"owners_count":25157135,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T19:41:46.482Z","updated_at":"2025-10-07T04:21:51.532Z","avatar_url":"https://github.com/THM-Graphs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GenAI Stack @ THM\n\nThis project is based on [Neo4j Genai Stack](https://github.com/docker/genai-stack) which bundles Neo4j, Langchain, Ollama and Streamlit into a docker compose environment.\nWe amended a bunch of changes to stay up to date with library upgrades and also to support multiple databases. \n\n## Installation\n\n### GPU\n\nDepending on the type of GPU the installation slightly differs. \nTechnically this is handled by [Docker Compose service profiles](https://docs.docker.com/compose/profiles/#start-specific-profiles).\n\n#### Nvidia GPU\n\nInstall [Nvidia's container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). On a Ubuntu/Debian based os run the following commands\n\n```\ncurl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \\\n  \u0026\u0026 curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \\\n    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \\\n    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list\nsudo apt-get update\nsudo apt-get install -y nvidia-container-toolkit\nsudo nvidia-ctk runtime configure --runtime=docker\nsudo nvidia-ctk config --set nvidia-container-cli.no-cgroups --in-place\n```\n\nAfter a full reboot, check that nvidia gpus are detected properly using `nvidia-smi`.\n\nEnsure you have set `export COMPOSE_PROFILES=linux-gpu-nvidia`, e.g. in `~/.profile`.\n\n#### AMD GPU\n\nEnsure you have set `export COMPOSE_PROFILES=linux-gpu-amd`, e.g. in `~/.profile`. \nDepending on your GPU you might need to tweak `HSA_OVERRIDE_GFX_VERSION`.\n\n#### On a Mac computer\n\nIn case you're using OLLAMA install it directly on the machine. \nDownload the model with `ollama pull llama3.2` and start OLLAMA with `ollama serve`.\n\n### configuration files\n\nOnce the GPU dependent part is done, copy `env.example` to `.env` and configure it to your needs.\nIt's important to set the uid/gid to the user running the Docker commands. \nYou might want to use the `id` command to figure out your uid/gid.\nThe first two settings `LLM` and `EMBEDDING_MODEL` should be set according to your needs.\n`NEO4J_PASSWORD` should be set to a reasonable password.\nNote that the password is only applied if the `data` folder is empty.\nYou cannot change the database password by simply changing it in `.env`.\nInstead follow the procedure from [Recover admin user and password](https://neo4j.com/docs/operations-manual/current/authentication-authorization/password-and-user-recovery/).\n\nFor authentication of the `loader` container copy `build-context/auth.yaml.example` to `build-context/auth.yaml` and apply your changes - esp. change the password.\n\n### download database dump\n\nThe database dump for regesta imperii is not part of the github repository, therefore download it:\n\n```bash\ncd backups\n./download.sh\n```\n\n### seeding the database\n\nTo have an inital dataset for neo4j, you can use a dump files.\nSince we're using the community edition of Neo4j the database name is fixed to `neo4j`.\n\n\u003e [!WARNING]\n\u003e Be aware that seeding will not happen if the respective database already exists.\n\n## start the containers\n\nUse `docker compose up -d --build` to build and run the containers.\nWith the usual suspect `docker compose ps` and `docker compose logs` progress can be monitored.\n\n## usage\n\nThe `bot` container is a interactive chat bot application using streamlit.\n`loader` is a container to vectorize the graph data.\nThis is intended to be a one-shot operation.\nThe `api` container exposes the search as a REST interface.\nA swagger UI for API documentation is available as well.\n\n|  | regestaimperii |\n| --- | --- |\n| bot | http://localhost:8601 |\n| loader | http://localhost:8602 |\n| api | http://localhost:8604/docs |\n\n\u003e [!NOTE]\n\u003e The `loader` container is password protected, the password is located in your `auth.yml` file.\n\n## development hints\n\n### hot reloading\n\nThe [docker-compose.yml](docker-compose.yml) does have support for [file watch](https://docs.docker.com/compose/how-tos/file-watch/).\nTo use it, run `docker compose up --watch --build --remove-orphans`.\n\n### remote debugging\n\nThe project supports remote debugging. \nTo use remote debugging install [Python Debugger](https://marketplace.visualstudio.com/items?itemName=ms-python.debugpy) to your VS code.\nTo switch on listening for a remote debugger use `DEBUG=true` in your `.env` file.\nIf set the debugger exposes port 5678 which the [launch.json](launch.json) file uses.\n\n***\n\nOriginal readme below:\n\n# GenAI Stack\nThe GenAI Stack will get you started building your own GenAI application in no time.\nThe demo applications can serve as inspiration or as a starting point.\nLearn more about the details in the [technical blog post](https://neo4j.com/developer-blog/genai-app-how-to-build/).\n\n# Configure\n\nCreate a `.env` file from the environment template file `env.example`\n\nAvailable variables:\n| Variable Name          | Default value                      | Description                                                             |\n|------------------------|------------------------------------|-------------------------------------------------------------------------|\n| OLLAMA_BASE_URL        | http://host.docker.internal:11434  | REQUIRED - URL to Ollama LLM API                                        |   \n| NEO4J_URI              | neo4j://database:7687              | REQUIRED - URL to Neo4j database                                        |\n| NEO4J_USERNAME         | neo4j                              | REQUIRED - Username for Neo4j database                                  |\n| NEO4J_PASSWORD         | password                           | REQUIRED - Password for Neo4j database                                  |\n| LLM                    | llama2                             | REQUIRED - Can be any Ollama model tag, or gpt-4 or gpt-3.5 or claudev2 |\n| EMBEDDING_MODEL        | sentence_transformer               | REQUIRED - Can be sentence_transformer, openai, aws, ollama or google-genai-embedding-001|\n| AWS_ACCESS_KEY_ID      |                                    | REQUIRED - Only if LLM=claudev2 or embedding_model=aws                  |\n| AWS_SECRET_ACCESS_KEY  |                                    | REQUIRED - Only if LLM=claudev2 or embedding_model=aws                  |\n| AWS_DEFAULT_REGION     |                                    | REQUIRED - Only if LLM=claudev2 or embedding_model=aws                  |\n| OPENAI_API_KEY         |                                    | REQUIRED - Only if LLM=gpt-4 or LLM=gpt-3.5 or embedding_model=openai   |\n| GOOGLE_API_KEY         |                                    | REQUIRED - Only required when using GoogleGenai LLM or embedding model google-genai-embedding-001|\n| LANGCHAIN_ENDPOINT     | \"https://api.smith.langchain.com\"  | OPTIONAL - URL to Langchain Smith API                                   |\n| LANGCHAIN_TRACING_V2   | false                              | OPTIONAL - Enable Langchain tracing v2                                  |\n| LANGCHAIN_PROJECT      |                                    | OPTIONAL - Langchain project name                                       |\n| LANGCHAIN_API_KEY      |                                    | OPTIONAL - Langchain API key                                            |\n\n## LLM Configuration\nMacOS and Linux users can use any LLM that's available via Ollama. Check the \"tags\" section under the model page you want to use on https://ollama.ai/library and write the tag for the value of the environment variable `LLM=` in the `.env` file.\nAll platforms can use GPT-3.5-turbo and GPT-4 (bring your own API keys for OpenAI models).\n\n**MacOS**\nInstall [Ollama](https://ollama.ai) on MacOS and start it before running `docker compose up` using `ollama serve` in a separate terminal.\n\n**Linux**\nNo need to install Ollama manually, it will run in a container as\npart of the stack when running with the Linux profile: run `docker compose --profile linux up`.\nMake sure to set the `OLLAMA_BASE_URL=http://llm:11434` in the `.env` file when using Ollama docker container.\n\nTo use the Linux-GPU profile: run `docker compose --profile linux-gpu up`. Also change `OLLAMA_BASE_URL=http://llm-gpu:11434` in the `.env` file.\n\n**Windows**\nOllama now supports Windows. Install [Ollama](https://ollama.ai) on Windows and start it before running `docker compose up` using `ollama serve` in a separate terminal. Alternatively, Windows users can generate an OpenAI API key and configure the stack to use `gpt-3.5` or `gpt-4` in the `.env` file.\n# Develop\n\n\u003e [!WARNING]\n\u003e There is a performance issue that impacts python applications in the `4.24.x` releases of Docker Desktop. Please upgrade to the latest release before using this stack.\n\n**To start everything**\n```\ndocker compose up\n```\nIf changes to build scripts have been made, **rebuild**.\n```\ndocker compose up --build\n```\n\nTo enter **watch mode** (auto rebuild on file changes).\nFirst start everything, then in new terminal:\n```\ndocker compose watch\n```\n\n**Shutdown**\nIf health check fails or containers don't start up as expected, shutdown\ncompletely to start up again.\n```\ndocker compose down\n```\n\n# Applications\n\nHere's what's in this repo:\n\n| Name | Main files | Compose name | URLs | Description |\n|---|---|---|---|---|\n| Support Bot | `bot.py` | `bot` | http://localhost:8501 | Main usecase. Fullstack Python application. |\n| Stack Overflow Loader | `loader.py` | `loader` | http://localhost:8502 | Load SO data into the database (create vector embeddings etc). Fullstack Python application. |\n| PDF Reader | `pdf_bot.py` | `pdf_bot` | http://localhost:8503 | Read local PDF and ask it questions. Fullstack Python application. |\n| Standalone Bot API | `api.py` | `api` | http://localhost:8504 | Standalone HTTP API streaming (SSE) + non-streaming endpoints Python. |\n| Standalone Bot UI | `front-end/` | `front-end` | http://localhost:8505 | Standalone client that uses the Standalone Bot API to interact with the model. JavaScript (Svelte) front-end. |\n\nThe database can be explored at http://localhost:7474.\n\n## App 1 - Support Agent Bot\n\nUI: http://localhost:8501\nDB client: http://localhost:7474\n\n- answer support question based on recent entries\n- provide summarized answers with sources\n- demonstrate difference between\n    - RAG Disabled (pure LLM response)\n    - RAG Enabled (vector + knowledge graph context)\n- allow to generate a high quality support ticket for the current conversation based on the style of highly rated questions in the database.\n\n![](.github/media/app1-rag-selector.png)\n*(Chat input + RAG mode selector)*\n\n|  |  |\n|---|---|\n| ![](.github/media/app1-generate.png) | ![](.github/media/app1-ticket.png) |\n| *(CTA to auto generate support ticket draft)* | *(UI of the auto generated support ticket draft)* |\n\n---\n\n##  App 2 - Loader\n\nUI: http://localhost:8502\nDB client: http://localhost:7474\n\n- import recent Stack Overflow data for certain tags into a KG\n- embed questions and answers and store them in vector index\n- UI: choose tags, run import, see progress, some stats of data in the database\n- Load high ranked questions (regardless of tags) to support the ticket generation feature of App 1.\n\n\n\n\n|  |  |\n|---|---|\n| ![](.github/media/app2-ui-1.png) | ![](.github/media/app2-model.png) |\n\n## App 3 Question / Answer with a local PDF\nUI: http://localhost:8503  \nDB client: http://localhost:7474\n\nThis application lets you load a local PDF into text\nchunks and embed it into Neo4j so you can ask questions about\nits contents and have the LLM answer them using vector similarity\nsearch.\n\n![](.github/media/app3-ui.png)\n\n## App 4 Standalone HTTP API\nEndpoints: \n  - http://localhost:8504/query?text=hello\u0026rag=false (non streaming)\n  - http://localhost:8504/query-stream?text=hello\u0026rag=false (SSE streaming)\n\nExample cURL command:\n```bash\ncurl http://localhost:8504/query-stream\\?text\\=minimal%20hello%20world%20in%20python\\\u0026rag\\=false\n```\n\nExposes the functionality to answer questions in the same way as App 1 above. Uses\nsame code and prompts.\n\n## App 5 Static front-end\nUI: http://localhost:8505\n\nThis application has the same features as App 1, but is built separate from\nthe back-end code using modern best practices (Vite, Svelte, Tailwind).  \nThe auto-reload on changes are instant using the Docker watch `sync` config.  \n![](.github/media/app5-ui.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthm-graphs%2Fgenai-stack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthm-graphs%2Fgenai-stack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthm-graphs%2Fgenai-stack/lists"}