https://github.com/singhdivyank/llama2

Using open-source Llama2 as an alternate to OpenAI
https://github.com/singhdivyank/llama2

gradio langchain llama-index llama2

Last synced: 18 days ago
JSON representation

Using open-source Llama2 as an alternate to OpenAI

• Host: GitHub
• URL: https://github.com/singhdivyank/llama2
• Owner: singhdivyank
• Created: 2023-08-03T16:33:02.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-08-10T14:03:19.000Z (over 1 year ago)
• Last Synced: 2025-01-01T09:32:31.283Z (23 days ago)
• Topics: gradio, langchain, llama-index, llama2
• Language: Python
• Homepage:
• Size: 176 KB
• Stars: 2
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# Using Llama2 as an alternate to ChatGPT for question answering over a file

Meta released open source [Llama2](https://ai.meta.com/llama/) as an LLM model, this repository attempts to utilize it for question answering over a pdf/word/CSV file instead of using ChatGPT API

There are two files- `download_version.py` and `using_llama.py`

* In the former a quantized Llama2 model is downloaded and implemented using CTransformer and LangChain

* In the later Llama2 model from Meta repository and implemented as HuggingFaceLLM using Llama_index framework

## Flow

1. Upload document

2. Create embeddings

3. Store in vector store (FAISS/ SimpleDirectoryReader)

4. User query

5. Create embeddings

6. Fetch documents

7. Send this to Llama

## Requirements

1. Langchain (designing prompts, history retention, and chains)

2. llama_index (for HuggingFace implementation)

3. FAISS (vector store)

4. Transformer for embeddings (HuggingFace `Sentence Transformer`)

5. Llama2 (serves as LLM)

6. Gradio (UI)

## Instructions

`pip install -r requirements.txt`

**For using a downloaded llama2 quantized model (downloaded_version.py)**: 

1. Download a Llama2 model from- https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main

2. update model code in line 29: `self.llama_model = "<'downloaded_model_name'>"`

3. `python3 download_version.py`

4. Upload a .pdf/.csv/.docx file and type in your question on the Gradio UI

Note: there are many available models, you can choose any as per the `model card`

**For using Llama as HuggingFaceLLm (using_llama.py)**:

1. get access to `meta-llama`

2. create a HuggingFace access token to access the repo

3. shortlist any llama2 model offered, and update line 28: `self.model_name = "<'llama2_model_name'>"`

4. `export HF_KEY = <'huggingface_access_token'>`

5. `python3 using_llama.py`

6. Upload a .pdf/.csv/.docx file and type in your question on the Gradio UI

Note: currently implemented model, `Llama-2-7b-chat-hf`

**A word of caution: this implementation consumes time and is memory expensive**

## Sample UI

![](image.png)