{"id":22694642,"url":"https://github.com/sachs7/multi-modal-langchain-chatbot","last_synced_at":"2025-04-12T07:38:21.970Z","repository":{"id":223890354,"uuid":"761818461","full_name":"sachs7/multi-modal-langchain-chatbot","owner":"sachs7","description":"A Multi-modal chatbot with LangChain, that supports RAG, Paperswithcode, and Image generation using Dall-E-3","archived":false,"fork":false,"pushed_at":"2024-02-28T16:10:27.000Z","size":2816,"stargazers_count":3,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T02:51:09.477Z","etag":null,"topics":["chatbot","dall-e-3","langchain","langchain-agent","multi-modal","openai-chatbot","paperswithcode","rag"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sachs7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-22T14:50:04.000Z","updated_at":"2024-06-02T22:19:23.000Z","dependencies_parsed_at":"2024-02-28T17:40:59.179Z","dependency_job_id":null,"html_url":"https://github.com/sachs7/multi-modal-langchain-chatbot","commit_stats":null,"previous_names":["sachs7/multi-modal-langchain-chatbot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sachs7%2Fmulti-modal-langchain-chatbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sachs7%2Fmulti-modal-langchain-chatbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sachs7%2Fmulti-modal-langchain-chatbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sachs7%2Fmulti-modal-langchain-chatbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sachs7","download_url":"https://codeload.github.com/sachs7/multi-modal-langchain-chatbot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248536119,"owners_count":21120681,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","dall-e-3","langchain","langchain-agent","multi-modal","openai-chatbot","paperswithcode","rag"],"created_at":"2024-12-10T03:09:01.506Z","updated_at":"2025-04-12T07:38:21.950Z","avatar_url":"https://github.com/sachs7.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multi-Modal LangChain Chatbot\n\nA Multi-Modal Chatbot with LangChain that also supports the agentic RAG\n\nThe ChatBot supports:\n\n1. Chat with your documents using basic RAG\n2. Query \"Paperswithcode\" site to get a gist of recent papers on any topic\n3. Generate images using DallE-3\n\nThe code base has been organized into the following folders:\n\n1. `docs`: Store your PDF documents in this folder\n2. `services`: Add all your agent-specific services here\n3. `tools`: Create a tool for your services\n\nThe `main.py` consists of the actual System Prompt and the registration of the tools.\nMake sure to register your new tools here. The UI can be created using both, Gradio, and Streamlit.\n\nNote: When using image generation, I was finding it difficult to represent images in Gradio, hence I went with Streamlit at the end. But feel free to play around.\n\n# How to run:\n\n1. `pip install -r requirements.txt` to install all the libraries\n2. From your terminal run, `export OPENAI_API_KEY=sk-xxxx`\n3. Store your PDFs in the `docs` folder\n4. If using Gradio, `python3 main.py`\n5. If using Streamlit, `streamlit run main.py`\n6. Once you start the app, a `chroma_persists` folder will be created along with a `logs` folder\n\n### Improvements:\n\n1. This is a basic RAG, one can create CRAG, Rank based RAG, etc\n2. Showcase Chat History (currently, not showing it)\n3. Add more agents specific to your needs (don't forget to register it in `main.py`)\n4. Use the `config` file to store model details, etc that are needed for RAG as well as other agents\n5. ~~Currently, the chromaDB gets recreated for every query to RAG. Need to handle this to make ChromaDB persist (instantiate once) once the service is up.~~ Fixed ✔️\n\n# Sample Output:\n\n1. ![Capabilities](images/1-capabilities.png)\n\n2. ![RAG Agent](images/2-rag-agent.png)\n\n3. ![PaperswithCode](images/3-paperswithcode-agent.png)\n\n4. ![Dall-e-3](images/4-dalle-3-agent.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsachs7%2Fmulti-modal-langchain-chatbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsachs7%2Fmulti-modal-langchain-chatbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsachs7%2Fmulti-modal-langchain-chatbot/lists"}