https://github.com/v-ade-r/llm-with-rag-full-opensource
LLM with RAG example for querying URLs or PDFs with Ollama and Gradio/Flask
https://github.com/v-ade-r/llm-with-rag-full-opensource
flask gradio llm ollama rag
Last synced: 21 days ago
JSON representation
LLM with RAG example for querying URLs or PDFs with Ollama and Gradio/Flask
- Host: GitHub
- URL: https://github.com/v-ade-r/llm-with-rag-full-opensource
- Owner: v-ade-r
- Created: 2024-05-10T16:00:58.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-05-12T23:48:35.000Z (about 1 year ago)
- Last Synced: 2025-02-14T01:48:29.542Z (4 months ago)
- Topics: flask, gradio, llm, ollama, rag
- Language: Python
- Homepage:
- Size: 1.6 MB
- Stars: 3
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM-with-RAG-full-opensource
This code enables you to utilize opensource LLMs locally with the RAG function. You can feed the LLM model with the data from a URL or PDF, resulting in improved answers as the LLM's inference is mostly based on the supplied data.
It wasn't easy to find a free reliable tutorial on this topic, especially when I wanted to use only opensource and run it locally, so that's why I decided to share this code. Finally, I found a super effective sollution here: https://www.youtube.com/watch?v=jENqvjpkwmw&list=LL&index=3. My code is essentially the same at its core. However, I made some small tweaks here and there, modified the Gradio part a bit, and added the Flask part.
**General step by step tutorial:**
1. Download Ollama (Ollama allows you to run open-source large language models locally) from https://ollama.com/, and install it.
2. Go to https://ollama.com/models, and find the name of a model you want to try, and which your hardware will be able to handle. "mistral" 7b or "llama3" are always a good deafault bets. "mwiewior/bielik" for polish language only.
3. Download a model by openning command line and typing: Ollama pull mistral. Or swap mistral for a model name of your choice.
4. Install needed packages.
**Querying URL with/without RAG:**\
5. Just follow the code.
**Querying PDF:**\
5. Just follow the code. Put pdf file in your project folder, or supply adequate path to it for PyPDFLoader.
**Using Gradio:**\
5. Just follow the code. Run the code, and put http://127.0.0.1:7860 in browser to test it locally, or add in .launch(share=True) to share the generated link with others.
**Using Flask:**\
5. Create in your project folder, a folder for temporarily storing PDFs.\
6. Create in your project folder, a folder named templates.\
7. In templates folder create a file named index3.html and setup everything there or just download mine. A few hints about customizations are even there.\
8. Just follow the code.\
9. Run the code, go to http://127.0.0.1:5000, and test it.\
**Problems to solve in the future:**
- It's very slow for big PDF files (around few minutes)
- It doesn't handle well PDFs with a lot of tables, pictures and strange objects.
- After few questions usually the restart is needed.