https://github.com/benitomartin/rag-azure-http
RAG Deployment with Azure Functions
https://github.com/benitomartin/rag-azure-http
azure-functions langchain openai python qdrant ruff
Last synced: 4 months ago
JSON representation
RAG Deployment with Azure Functions
- Host: GitHub
- URL: https://github.com/benitomartin/rag-azure-http
- Owner: benitomartin
- Created: 2024-07-14T09:22:41.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-08-17T08:12:45.000Z (almost 2 years ago)
- Last Synced: 2025-10-07T21:57:35.694Z (9 months ago)
- Topics: azure-functions, langchain, openai, python, qdrant, ruff
- Language: Python
- Homepage:
- Size: 1.08 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# RAG AZURE HTTP QDRANT 🧑💻
This repository contains a full Q&A pipeline using LangChain framework, Qdrant as vector database and Azure Function with HttpTrigger. The data used are research papers that can be loaded into the vector database, and the Azure Functions processes the request using the retrieval and generation logic. Therefore it can use any other research paper from Arxiv.
The main steps taken to build the RAG pipeline can be summarized as follows:
* **Data Ingestion**: load data from https://arxiv.org
* **Indexing**: RecursiveCharacterTextSplitter for indexing in chunks
* **Vector Store**: Qdrant inserting metadata
* **QA Chain Retrieval**: RetrievalQA
* **Azure Function**: Process the request with HttpTrigger
Feel free to ⭐ and clone this repo 😉
## Tech Stack
## Project Structure
The project has been structured with the following files:
- `.env_sample`: sample environmental variables
- `Makefile`: install requirements, formating, linting, and clean up
- `pyproject.toml`: linting and formatting using ruff
- `requirements.txt:` project requirements
- `create_vector_store.py:` script to create the collection in Qdrant
- `function_app.py:` Azure function app
- `rag_app.py:` RAG Logic
- `host.json:` metadata file with configuration options for the function app for deployment
- `local.settings.json`: metadata file with configuration options for the function app locally
- `deploy.sh:` script to create and publish the function
## Project Set Up
The Python version used for this project is Python 3.10. You can follow along the medium article.
1. Clone the repo (or download it as a zip file):
```bash
git clone https://github.com/benitomartin/rag-azure-http.git
```
2. Create the virtual environment named `main-env` using Conda with Python version 3.10:
```bash
conda create -n main-env python=3.10
conda activate main-env
```
3. Execute the `Makefile` script and install the project dependencies included in the requirements.txt:
```bash
pip install -r requirements.txt
or
make install
```
4. Create **Azure Account** and install **Azure CLI** and **Functions Core Tools**.
5. Test the function locally
```bash
func start
```
```bash
curl -X POST "http://localhost:7071/api/req" \
-H "Content-Type: application/json" \
-d '{"query": "positional encoding"}'
```
6. Create and publish the App: Make sure the `.env` file is complete and run the `deploy.sh` script
```bash
chmod +x deploy.sh
./deploy.sh
```
```bash
curl -X POST https://app-rag-function.azurewebsites.net/api/req \
-H "Content-Type: application/json" \
-d '{"query": "Positional Encoding"}'
```
