Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/Ce11an/surrealdb-openai
Example of RAG using SurrealDB and OpenAI
https://github.com/Ce11an/surrealdb-openai
fastapi htmx jinja2 openai python surrealdb
Last synced: about 1 month ago
JSON representation
Example of RAG using SurrealDB and OpenAI
- Host: GitHub
- URL: https://github.com/Ce11an/surrealdb-openai
- Owner: Ce11an
- License: mit
- Created: 2024-02-28T17:34:25.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-09-04T21:11:43.000Z (4 months ago)
- Last Synced: 2024-10-27T19:40:53.990Z (about 2 months ago)
- Topics: fastapi, htmx, jinja2, openai, python, surrealdb
- Language: Python
- Homepage:
- Size: 50.8 KB
- Stars: 22
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-surreal - SurrealDB x OpenAI - Example of RAG using SurrealDB and OpenAI. (Projects)
README
# SurrealDB x OpenAI: A Chat Playground
Welcome to a project that explores the synergy between SurrealDB and OpenAI.
https://github.com/Ce11an/surrealdb-openai/assets/60790416/a4e6a967-321a-4ca0-8f64-1687512aab38
## Project Overview
This project aims to demonstrate the potential of integrating SurrealDB with OpenAI by importing 25,000 Wikipedia articles, complete with vector embeddings generated by OpenAI. The ultimate goal is to develop a Retrieval-Augmented Generation (RAG) question-answering system.
The backend is powered by a FastAPI server, with Jinja2 handling template rendering and htmx enriching the frontend chat interface.
## Getting Started
### Prerequisites
1. **SurrealDB**: Ensure that SurrealDB is installed and configured on your machine. Installation instructions are available [here](https://surrealdb.com/install).
2. **Python 3.11**: We use [uv](https://docs.astral.sh/uv/) to manage dependencies.
3. **SurrealDB Python SDK**: The [SurrealDB Python SDK](https://github.com/surrealdb/surrealdb.py/tree/main) is currently in beta. To install the dependency, please ensure you have `rust` installed on your system.
### Installation
1. Clone the repository.
2. You will need an OpenAI API token for this project. If you don't have one, you can obtain it by following the [OpenAI Developer Quickstart](https://platform.openai.com/docs/quickstart). Rename the `.env.example` file to `.env` and replace the placeholder with your key.
**Note:** This project is intended for experimental purposes only. Ensure that your OpenAI API key is kept secure.
### Setting Up SurrealDB
To initialise SurrealDB, use the following `make` commands:
1. Start SurrealDB with on-disk persistence:
```bash
make surreal-start
```
2. Initialise the database with the required tables and functions:
```bash
make surreal-init
```
3. If you need to reset the database:
```bash
make surreal-remove
```
### Python Environment
With `uv` installed, run:
```bash
uv sync
```
Activate the Python virtual environment:
```bash
source .venv/bin/activate
```
### Downloading the Dataset
Download the Simple English Wikipedia dataset, which includes vector embeddings (~700MB compressed, ~1.7GB CSV):
```bash
download-data
```
### Populating SurrealDB
Insert the dataset into SurrealDB:
```bash
surreal-insert
```
### Starting the Chat Application
To start the chat server:
```bash
make server-start
```
## Support
If you find this project helpful, consider supporting the development:
