https://github.com/v-ade-r/chatbot-based-on-contextual-rag-full-opensource
Chatbot based on Contextual RAG with Hybrid Search and Reranking with short conversation history awareness, fully OpenSource.
https://github.com/v-ade-r/chatbot-based-on-contextual-rag-full-opensource
chatbot contextual-retrieval fastapi hybrid-search llm ollama rag reranking
Last synced: about 2 months ago
JSON representation
Chatbot based on Contextual RAG with Hybrid Search and Reranking with short conversation history awareness, fully OpenSource.
- Host: GitHub
- URL: https://github.com/v-ade-r/chatbot-based-on-contextual-rag-full-opensource
- Owner: v-ade-r
- Created: 2025-01-20T23:20:26.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-01-27T01:04:14.000Z (4 months ago)
- Last Synced: 2025-02-14T01:48:21.266Z (3 months ago)
- Topics: chatbot, contextual-retrieval, fastapi, hybrid-search, llm, ollama, rag, reranking
- Language: Python
- Homepage:
- Size: 666 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Chatbot based on Contextual RAG with Hybrid Search
Chatbot with Conversation History Awareness based on Contextual RAG with Hybrid Search and Reranking, fully OpenSource.
## **Justification of this code**
The idea was to create a fully functional, completely open source Contextual RAG with Hybrid Search and Reranking application.
## Some idea and code explanations
**Todo
**
Front is tragic, but I don't really care, because it's not a front project.
## **Usage tips**
1. Download and install Ollama
2. (Download the Llama3.1 model) In command line type: ollama run llama3.1
3. In command line type: Ollama serve
4. Download Docker and set it up.
5. Open Docker Desktop
6. In cmd type: docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:8.8.0
7. Good to go. Let's evaluate the retrieving capabilities running evaluate.py!
8. Or run the app.py and use it. I would suggest you to trimm this codebase json to only a few chunks, create a database and upload your pdf to test the app. Remember to change the name of the database!!
## **Retrieval evaluation**
The current code is capable of evaluating the approach using data structurized like the data prepared by Anthropic.
I evaluated the retrieving accuracy at each stage. Pass@n - represents the accuracy of getting the 'golden chunk' (most relevant chunk for the query) within the top-n (top5 and top20) retrieved chunks.
VectorDB (only semantic search):
Pass@5: 63.76%
Pass@20: 79.66%
ContextualVectorDB:
Pass@5: 69.84%
Pass@20: 83.13%
ContextualVectorDB + BM25 (adding BM25 creates Hybrid Search):
Pass@5: 76.53%
Pass@20: 87.37%
ContextualVectorDB + BM25 + Reranker:
Pass@5: 81.32%
Pass@20: 90.83%
------------------
(Context created by GPT-4o-mini)
Contextual + BM25 + reranker:
Pass@5: 81.99%
Pass@20: 93.75%
## References
https://github.com/anthropics/anthropic-cookbook/blob/main/skills/contextual-embeddings/guide.ipynb