{"id":18395833,"url":"https://github.com/deepset-ai/rag-with-nvidia-nims","last_synced_at":"2025-04-07T03:35:12.490Z","repository":{"id":240969813,"uuid":"797666517","full_name":"deepset-ai/rag-with-nvidia-nims","owner":"deepset-ai","description":"🚀 Use NVIDIA NIMs with Haystack pipelines","archived":false,"fork":false,"pushed_at":"2024-09-04T12:01:26.000Z","size":647,"stargazers_count":30,"open_issues_count":2,"forks_count":8,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-22T13:22:17.390Z","etag":null,"topics":["ai","deployment","haystack","llm"],"latest_commit_sha":null,"homepage":"https://haystack.deepset.ai/blog/haystack-nvidia-nim-rag-guide","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepset-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-08T09:30:23.000Z","updated_at":"2025-01-14T04:27:29.000Z","dependencies_parsed_at":"2024-09-05T12:53:18.802Z","dependency_job_id":"9d996722-88ae-453e-8740-2755d3049661","html_url":"https://github.com/deepset-ai/rag-with-nvidia-nims","commit_stats":null,"previous_names":["deepset-ai/nvidia-haystack","deepset-ai/rag-with-nvidia-nims"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Frag-with-nvidia-nims","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Frag-with-nvidia-nims/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Frag-with-nvidia-nims/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Frag-with-nvidia-nims/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepset-ai","download_url":"https://codeload.github.com/deepset-ai/rag-with-nvidia-nims/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247589793,"owners_count":20963022,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deployment","haystack","llm"],"created_at":"2024-11-06T02:12:02.074Z","updated_at":"2025-04-07T03:35:12.199Z","avatar_url":"https://github.com/deepset-ai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Build Air-Gapped RAG with Nvidia NIMs and Haystack\n\n📚 This repository is accompanied by our article [\"Building RAG Applications with NVIDIA NIM and Haystack on K8s\"](https://haystack.deepset.ai/blog/haystack-nvidia-nim-rag-guide)\n\n\u003e Info: This repo is set up to use models hosted and accessible via https://build.nvidia.com/ \n\u003e\n\u003e These models are already available and you can use them by creating yourself API keys through the platform.\n\u003e The project is set up so that you can change these models to NIM deployments by setting the `model` name and `api_url` in the `NvidiaGenerator`, `NvidiaDocumentEmbedder` and `NvidiaTextEmbedder` components.\n\u003e \n\u003e 👩🏻‍🍳 We also provide a notebook on Haystack Cookbooks that provide the same code and setup, only expecting self-hosted NIMs\n\u003e \n\u003e \u003ca href=\"https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/rag-with-nims.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e\n\n## Run with Docker\n\n1. `pip install -r requirements.txt`\n2. Create a `.env` file and add `NVIDIA_API_KEY` (if you're using hosted models via https://build.nvidia.com/)\n3. `docker-compose up`\n6. `hayhooks deploy rag.yaml`\n7. Go to `localhost:1416/docs` to interact with your RAG pipeline\n\n## File Structure\n\n- `indexing.py`: This script preproecesses, embeds and writes ChipNemo.pdf into a Qdrant database\n- `rag.py`: This scripts runs a RAG pipeline with a NIM LLM and retrieval model. \n- `Dockerfile`: This is used by the docker-compose file to install dependencies\n- `docker-compose.yml`: This is the docker compose file we use to spin up a container for hayhooks (Haystack pipeline deployment) and Qdrant\n- `rag.yaml`: This is the serialized RAG pipeline which is the same as `rag.py` in YAML. We use this to deploy our pipeline with hayhooks\n-  \u003ca href=\"https://colab.research.google.com/github/deepset-ai/haystack-cookbook/blob/main/notebooks/rag-with-nims.ipynb\" target=\"_parent\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e: This notebook shows you how you can set up your components to use self-hosted NIMs.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Frag-with-nvidia-nims","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepset-ai%2Frag-with-nvidia-nims","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Frag-with-nvidia-nims/lists"}