{"id":20685375,"url":"https://github.com/yas-sim/openvino-chatbot-rag-pdf","last_synced_at":"2025-04-22T13:40:31.556Z","repository":{"id":219121225,"uuid":"735585145","full_name":"yas-sim/openvino-chatbot-rag-pdf","owner":"yas-sim","description":"LLM Chatbot-RAG by OpenVINO. The chatbot can read a PDF file and answer to the related questions.","archived":false,"fork":false,"pushed_at":"2024-01-29T12:34:37.000Z","size":107,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-08T09:47:03.195Z","etag":null,"topics":["chatbot","huggingface","large-language-models","llama","llama2","llm","neural-chat","openvino","pdf","python","rag","retrieval-augmented-generation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yas-sim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-25T12:53:09.000Z","updated_at":"2024-03-26T09:27:58.000Z","dependencies_parsed_at":"2024-01-29T14:45:06.041Z","dependency_job_id":null,"html_url":"https://github.com/yas-sim/openvino-chatbot-rag-pdf","commit_stats":{"total_commits":13,"total_committers":1,"mean_commits":13.0,"dds":0.0,"last_synced_commit":"074c312f797f59742261f0d20b9593895b50d901"},"previous_names":["yas-sim/openvino-chatbot-rag-pdf"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fopenvino-chatbot-rag-pdf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fopenvino-chatbot-rag-pdf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fopenvino-chatbot-rag-pdf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yas-sim%2Fopenvino-chatbot-rag-pdf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yas-sim","download_url":"https://codeload.github.com/yas-sim/openvino-chatbot-rag-pdf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249619210,"owners_count":21300997,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","huggingface","large-language-models","llama","llama2","llm","neural-chat","openvino","pdf","python","rag","retrieval-augmented-generation"],"created_at":"2024-11-16T22:27:06.817Z","updated_at":"2025-04-22T13:40:31.532Z","avatar_url":"https://github.com/yas-sim.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenVINO Chatbot using RAG (PDF)\r\n\r\n## Description\r\n\r\nThis project demonstrates how to extend the LLM models' capability to answer a question related to a given document.\r\nThe project consists of two programs. One is for preparation, and the other is for question and answering using LLM.\r\nThe preparation program will read a PDF file and generate a database (vector store).\r\nThe LLM model will pick up a collection of a fraction of the input document that is related to the given query from the user and then answer the query by referring to the picked-up documents. This technique is so called RAG (Retrieval Augmented Generation).\r\n\r\n## Programs/Files\r\n\r\n|#|file name|description|\r\n|---|---|---|\r\n|1|`vectorstore_generator.py`|Reads a PDF file and generates a vectorstore.\u003cbr\u003eYou can modify this program to make it read and use the other document file format in this RAG chatbot demo.|\r\n|2|`openvino-chatbot-rag-pdf.py`|LLM chatbot using OpenVINO. Answer to the query by refeering a vectorstore.|\r\n|3|`llm-model-downloader.py`|Downloads LLM models from HuggingFace and converts them into OpenVINO IR models.\u003cbr\u003eThis program downloads follosing models by default:\u003cbr\u003e* `dolly-v2-3b`\u003cbr\u003e* `neural-chat-7b-v3-1`\u003cbr\u003e* `tinyllama-1.1b-chat-v0.6`\u003cbr\u003e* `youri-7b-chat`.\u003cbr\u003eYou can download `llama2-7b-chat` by uncomment some lines in the code.|\r\n|4|`.env`|Some configurations (model name, model precision, inference device, etc)|\r\n\r\n## How to run\r\n0. Install prerequisites\r\n\r\n```sh\r\npython -m venv venv\r\nvenv\\Scripts\\activate\r\npython -m pip install -U pip\r\npip install -U setuptools wheel\r\npip install -r requirements.txt\r\n```\r\n\r\n1. Download LLM models\r\n\r\nThis program downloads the LLM models and converts them into OpenVINO IR models.\r\nIf you don't want to download many LLM models, you can comment out the models in the code to save time.\r\n```sh\r\nphthon llm-model-downloader.py\r\n```\r\n\r\n2. Preparation - Read a PDF file and generate a vectorstore\r\n\r\n```sh\r\npython vectorstore_generator.py -i input.pdf\r\n```\r\n`./vectorstore_{pdf_basename}` directory will be created. The data of the vectorstore will be stored in the directory. E.g. `./vectorstore_input`.\r\n![generation](./resources/generation.png)\r\n\r\n3. Run LLM Chatbot\r\n\r\n```sh\r\npython openvino-chatbot-rag-pdf.py -v vectorstore_input\r\n```\r\n![chatbot](./resources/chatbot.png)\r\n\r\n## Appendix - vectorstore (retriever) test tool\r\n\r\nYou can check which fraction of the input documents are picked up from the vectorstore based on the input query.\r\n```sh\r\npython test_vectorstore.py -v vectorstore_hoge\r\n```\r\n\r\n![test_vectorstore](./resources/test_vectorstore.png)\r\n## Test environment\r\n\r\n- Windows 11\r\n- OpenVINO 2023.2.0\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyas-sim%2Fopenvino-chatbot-rag-pdf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyas-sim%2Fopenvino-chatbot-rag-pdf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyas-sim%2Fopenvino-chatbot-rag-pdf/lists"}