{"id":22100458,"url":"https://github.com/joyceannie/medical_chatbot","last_synced_at":"2025-07-29T10:03:44.547Z","repository":{"id":227702795,"uuid":"772175436","full_name":"joyceannie/Medical_ChatBot","owner":"joyceannie","description":"The objective of this project is to create a chatbot that can be used to communicate with users to provide answers to their health issues. This is a RAG implementation using open source stack. ","archived":false,"fork":false,"pushed_at":"2024-03-14T18:00:23.000Z","size":3895,"stargazers_count":1,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-05T22:07:13.232Z","etag":null,"topics":["chatbot","fastapi","generative-ai","llm","nlp","python","qdrant","rag","vector-database"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/joyceannie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-14T17:16:09.000Z","updated_at":"2025-05-27T12:23:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"b063822c-fa21-4ba6-a19c-21af6be90633","html_url":"https://github.com/joyceannie/Medical_ChatBot","commit_stats":null,"previous_names":["joyceannie/medical_chatbot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/joyceannie/Medical_ChatBot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joyceannie%2FMedical_ChatBot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joyceannie%2FMedical_ChatBot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joyceannie%2FMedical_ChatBot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joyceannie%2FMedical_ChatBot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/joyceannie","download_url":"https://codeload.github.com/joyceannie/Medical_ChatBot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/joyceannie%2FMedical_ChatBot/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267668192,"owners_count":24124963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","fastapi","generative-ai","llm","nlp","python","qdrant","rag","vector-database"],"created_at":"2024-12-01T05:14:08.741Z","updated_at":"2025-07-29T10:03:44.517Z","avatar_url":"https://github.com/joyceannie.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Medical RAG App\n\nThe objective of this project is to create a chatbot that can be used to communicate with users to provide answers to their health issues. This is a RAG implementation using open source stack. The LLM used for the chatbot is [BioMistral](https://huggingface.co/BioMistral/BioMistral-7B). BioMistral is an opensource LLM finetuned for medical domains. In order to run the application locally on CPU, a [quantized model](https://huggingface.co/MaziyarPanahi/BioMistral-7B-GGUF) of the LLM is used. [PubMedBert](https://huggingface.co/NeuML/pubmedbert-base-embeddings) is the embedding model used for the application. This model is finetuned using sentence-transormers. It outperforms all other sentence transformer models for tasks on medical domain. It creates a 768 dimensional dense vector for embedding. [Qdrant](https://qdrant.tech/) is the vector database used for storing the vectors. Qudrant is a self hosted open source vector database. LangChain and Llama CPP are used as the orchestration framework.\n\n\n## Data\nThe `data` folder contains the data used for creating the vectors. I have used 2 pdf files. You can use any number of files. The data in these files will be be converted to vectors and stored in Qdrant. \n\n\n## Architecture\n\nThe initial step is to create a vector embedding of all the documents. This is done using the script `ingest.py`. When the user inputs a query, the vectors with the highest similarity are retrieved from the vector database. This is goven as the context for the LLM. THis would help to reduce the context length of the input. In order to facilitate memory, we are using ConversationalRetrivalChain. The previous query and output are passed to the LLM so that it can rewrite the new query. This gives more context to the LLM, and gives better responses. \n\n## How to Run\n\n\u003cb\u003eStep 1:Setup virtual environment\u003c/b\u003e\n\nIn order to run the application, you have to create a new Python virtual environment.\n\n```\npython3 -m venv venv\nsource venv venv\n```\n\n\u003cb\u003eStep 2: Install all the requirements\u003c/b\u003e\n\n```\npip install -r requirements.txt\n```\n\n\u003cb\u003eStep 3: Setup Qdrant\u003c/b\u003e\n\nI am using docker image of Qdrant. If you are following this method, you should install docker.\n\n```\ndocker pull qdrant/qdrant\n\ndocker run -p 6333:6333 -p 6334:6334 \\\n    -v $(pwd)/qdrant_storage:/qdrant/storage:z \\\n    qdrant/qdrant\n\n```\n\n\u003cb\u003eStep4: Download the quantized version of LLM\u003c/b\u003e\n\nDownaload the quantized version of BioMistral from [here](https://huggingface.co/MaziyarPanahi/BioMistral-7B-GGUF) to the project directory.\n\n\u003cb\u003eStep5: Ingest data to Qdrant collection\u003c/b\u003e\n\n```\npython3 ingest.py\n```\n\n\n\u003cb\u003eStep 6: Start the FastAPI server \u003c/b\u003e\n\n```\nuvicorn main:app\n```\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoyceannie%2Fmedical_chatbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjoyceannie%2Fmedical_chatbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjoyceannie%2Fmedical_chatbot/lists"}