{"id":22132090,"url":"https://github.com/olonok69/nim_llamaindex","last_synced_at":"2026-04-12T02:34:20.517Z","repository":{"id":261955355,"uuid":"885818508","full_name":"olonok69/Nim_LlamaIndex","owner":"olonok69","description":"Integracion LLamaIndex with NVIDIA NIM","archived":false,"fork":false,"pushed_at":"2024-11-10T13:04:34.000Z","size":2533,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-29T14:50:29.679Z","etag":null,"topics":["llamaindex","nvidia-nim","pymupdf","pymupdf4llm","python","rag","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/olonok69.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-09T13:32:44.000Z","updated_at":"2024-11-10T13:04:37.000Z","dependencies_parsed_at":"2025-01-29T14:47:28.551Z","dependency_job_id":"31190067-9480-4ab4-890d-4373f66e7a29","html_url":"https://github.com/olonok69/Nim_LlamaIndex","commit_stats":null,"previous_names":["olonok69/nim_llamaindex"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olonok69%2FNim_LlamaIndex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olonok69%2FNim_LlamaIndex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olonok69%2FNim_LlamaIndex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olonok69%2FNim_LlamaIndex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/olonok69","download_url":"https://codeload.github.com/olonok69/Nim_LlamaIndex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245241085,"owners_count":20583160,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llamaindex","nvidia-nim","pymupdf","pymupdf4llm","python","rag","streamlit"],"created_at":"2024-12-01T18:39:31.368Z","updated_at":"2026-04-12T02:34:15.484Z","avatar_url":"https://github.com/olonok69.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NVIDIA and LlamaIndex Developer Contest\nIn this repository I have the submission to the \"NVIDIA and LlamaIndex Developer Contest\", for reference https://developer.nvidia.com/llamaindex-developer-contest\n\n## Description\nI have created a small RAG application which allow to extract information from unstructured data. In this case from pdf files, but with few modifications we can extend its functionality to extract from other formats. The best description of this application is \"Speak with your Data\"\n\nMain Technolgies use in this application are:\n- Nvidia NIM \n- LLamaIndex\n- StreamLit\n- Docker\n- Python 3.11\n\n# Nvidia NIM\nNIM is a set of optimized cloud-native microservices designed to shorten time-to-market and simplify deployment of generative AI models anywhere, across cloud, data center, and GPU-accelerated workstations. It expands the developer pool by abstracting away the complexities of AI model development and packaging for production ‌using industry-standard APIs.\nhttps://developer.nvidia.com/blog/nvidia-nim-offers-optimized-inference-microservices-for-deploying-ai-models-at-scale/\n\nhttps://docs.api.nvidia.com/nim/reference/llm-apis\n\n![file](images/NIM.png)\n\n# LLamaIndex\nLlamaIndex is a framework for building context-augmented generative AI applications with LLMs including agents and workflows.\n\nhttps://docs.llamaindex.ai/en/stable/\n\n![file](images/llamaindex.png)\n\n# Models\n\n### Embeddings nvidia/embed-qa-4\n```\nArchitecture Type: Transformer\nNetwork Architecture: Fine-tuned E5-Large-Unsupervised retriever\nEmbedding Dimension: 1024\nParameter Count: 335 million\n```\nhttps://build.nvidia.com/nvidia/embed-qa-4/modelcard \n\n\n### LLM,s From NGC\nYou need to create an account in NGC and there access to NIM Dashboard and there generate an NVIDIA_API_KEY\nI have my Nvidia API key in a .env file and either in the APP or the Notebook I load it into the environment variables\n```\nfrom dotenv import dotenv_values\nimport os\n# read env file\nROOT_DIR = os.getcwd()\nconfig = dotenv_values(os.path.join(ROOT_DIR, \"keys\", \".env\"))\nos.environ['NVIDIA_API_KEY'] = config.get('NVIDIA_API_KEY')\n```\n\n*To access models just instantiate the objects NVIDIA and NVIDIAEmbedding after setting the NVIDIA_API_KEY to the object Settings of LLamaIndex*\n\nex:\n```\nfrom llama_index.core import Settings\nfrom llama_index.llms.nvidia import NVIDIA\nfrom llama_index.embeddings.nvidia import NVIDIAEmbedding\n\nSettings.embed_model = NVIDIAEmbedding(model=\"NV-Embed-QA\", truncate=\"END\")\n\n# Here we are using meta/llama-3.2-3b-instruct model from API Catalog\nSettings.llm = NVIDIA(model=\"meta/llama-3.2-3b-instruct\", temperature=0.7)\n\n```\n\n#### In Demo LLM Microsoft Phi3 3.5 128K\n\nPhi-3.5-mini is a lightweight, state-of-the-art open model built upon datasets used for Phi-3 - synthetic data and filtered publicly available websites - with a focus on very high-quality, reasoning dense data. The model belongs to the Phi-3 model family and supports 128K token context length. The model underwent a rigorous enhancement process, incorporating both supervised fine-tuning, proximal policy optimization, and direct preference optimization to ensure precise instruction adherence and robust safety measures.\n\nhttps://huggingface.co/microsoft/Phi-3.5-mini-instruct\n\nhttps://github.com/microsoft/Phi-3CookBook\n\n\n### Notebook Include LLama 3.2 3b Instruct\n\nThe Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks.\n\nhttps://huggingface.co/meta-llama/Llama-3.2-3B-Instruct\n\n#### Note. You need to get access to this model before use. In HugginFace Hub request access to Meta\n\n# Install \nBefore Start , you need to have installed Anaconda, Miniconda or other environments manager tool, Vscode and docker Desktop. If you plan to use a GPU in your docker Install Nvidia Container Toolkit (See https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). I am working in a Debian Distro under WSL 2 , host Windows 11\n\n1. Clone this Repository.\n1. Create a python Environment with python 3.11 and activate it\n2. If you want to follow the code in Jupyter install jupyter or Jupyter lab and open [this notebook](RAG_LlamaIndex_Nim.ipynb)\n3. Install requirements.txt. pip install -r requirements.txt\n4. to run the application on a shell : python -m streamlit run main.py\n6. If you want to create the docker container run docker build --pull --rm --build-arg user_app=app -f \"Dockerfile\" -t nim_llamaindex:latest \".\"\n\nNote: You need to create a folder with a .env file and a confg.yaml file.\n\n*.env*\n```\n\nINDEX_NAME=name_of_your_index\nNVIDIA_API_KEY=nvapi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\nNVIDIA_EMBEDDINGS=NV-Embed-QA\nNVIDIA_MODEL=microsoft/phi-3-small-128k-instruct\n\n```\n\n*config.yaml READ PROCEDURE at bottom of this file*\n```\ncredentials:\n  usernames:\n    user1:\n      email: test@test.com\n      name: user1 \n      password: hashed_password # To be replaced with hashed password\n    nvidiademo:\n      email: demo@test.com\n      name: nvidiademo\n      password: hashed_password # To be replaced with hashed password\n\n```\n\n# How to use it\n\n### Authentication Page\nIntroduce your User and Password . Follow the procedure in the bottom to create an user in Streamlit. For the demo I have created two users nvidiademo and user1. On the on the file hasher/ex.yaml file you have all details.\n\n\n![file](images/login.png)\n\n### Go to Speak pdf page\nClick on the left Menu speak pdf and page will open\n\n![file](images/speak_pdf.png)\n\n### Choose PDF to speak with\n\nClick in browse files in the Upload PDF file Component and the pdf visualizer will appears. You can scroll down and up to visualize the content of the pdf\n\n![alt text](images/browse.png)\n\n### Parse PDF\nClick on Parse PDF and this internally will use pymupdf4llm to convert each page of the document into LLamaIndex Documents. We are using the model \"NV-Embed-QA\" for embeddings. One we have created the embeddings docs they are stored in the VectorIndex object and persisted to the disk. If the file was already parsed and added to the VectorStore the application will not do anything, just use the VectorStore. The Input box to write your question will appears under the Parse Butom\n\n![alt text](images/input.png)\n\n### Ask the model Something about the document\nWrite in the input your question to the document and press Enter. The answer from the Model is written in the Text Area on the Right\nIn this example:\n```\nProvide Main Details of the company Aardvark Constructions Limited. Including following details:\nName:\nCountry:\nCompany Number:\nIncorporated:\nCompany Type:\nCompany Status:\nPrimary Addresses Registered Office:\nAccounting Dates:\nConfirmation Statement:\n```\n\n![alt text](images/speak.png)\n\nYou can ask as much as you want. As it has memory, its remember the company you are asking for\n\n```\nFrom Management Details extract:\nManaged By:\nManaged By Email:\n```\n\n![alt text](images/speak2.png)\n\n# Leave the Conversation\n\nClick on button Salir (Leave Conversation) above the Text area on the Right and it close  the current Conversation. The application will save your conversation on the folder answers ina txt file with the format filename_YYYY-MM-DD_H-M-S_history.txt\n\nExample this session\n\n![alt text](images/image.png)\n\n\n# TODO\n- Improve the control of files in the vectorstore\n- Maps volumen to folders in Docker Container\n- Include other formats and made the bot multimodal\n\n\n# How to create Users and Passwords to this app\n\ncreate a yaml file with name config.yaml in the keys folder\nhere you have and example with 2 users user1 and nvidiademo\n```\ncredentials:\n  usernames:\n    user1:\n      email: test@gmail.com\n      name: user1 \n      password: $2b$12$ZoeMqMyYQZqFGM7iF2bugeYG3E1kL7RA8jLkTvPcpdIlpIz46Yz/i # To be replaced with hashed password\n    nvidiademo:\n      email: demo@nvidia.com\n      name: nvidiademo\n      password: $2b$12$vL2J2PauSXISet6wZIM0j.DFZdq7cyrJ/2TQEn8UT.NyxZuKGfEZa # To be replaced with hashed password\ncookie:\n  expiry_days: 0\n  key: laPalma50 # Must be string\n  name: thecokkei\n  ```\n\n\n  ###### To generate the hashes\n  - Go to the folder hasher and edit the file ex.yaml\n  - Fill email , name and password with your values\n  - Run hasher.py\n  - Replace the password in config.yaml with its hash value\n\n  ```\n  credentials:\n  usernames:\n    user1:\n      email: test@gmail.com\n      name: user1 \n      password:  text#me\n    nvidiademo:\n      email: demo@nvidia.com\n      name: nvidiademo\n      password:  laPalma50 \ncookie:\n  expiry_days: 0\n  key: laPalma50 # Must be string\n  name: thecokkei\n  ```\n\n  OUTPUT\n\n  ```\n  {'usernames': {'user1': {'email': 'test@gmail.com', 'name': 'user1', 'password': '$2b$12$ZoeMqMyYQZqFGM7iF2bugeYG3E1kL7RA8jLkTvPcpdIlpIz46Yz/i'}, 'nvidiademo': {'email': 'demo@nvidia.com', 'name': 'nvidiademo', 'password': '$2b$12$vL2J2PauSXISet6wZIM0j.DFZdq7cyrJ/2TQEn8UT.NyxZuKGfEZa'}}}\n  ```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folonok69%2Fnim_llamaindex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Folonok69%2Fnim_llamaindex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folonok69%2Fnim_llamaindex/lists"}