https://github.com/makspll/chatbot_playground

playground repository with open AI
https://github.com/makspll/chatbot_playground

Last synced: about 1 month ago
JSON representation

playground repository with open AI

• Host: GitHub
• URL: https://github.com/makspll/chatbot_playground
• Owner: makspll
• Created: 2023-12-01T17:10:06.000Z (12 months ago)
• Default Branch: main
• Last Pushed: 2023-12-04T15:24:12.000Z (12 months ago)
• Last Synced: 2024-05-02T02:55:06.282Z (7 months ago)
• Language: Python
• Size: 81.1 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# Architecture

- user asks query

- query is run through first LLM prompt which generates context via valid SQL query

- query is improved by running the query and putting the SQL return values with the built query into another prompt for another LLM pass, where the model is asked to either fix any errors or improve the query

- this is done untill MAX_TRIES or the model returns something containing "valid"

- the output + query + user query are used combined into primary prompt which then generates the final response to the user

Pros:

- Great against hallucinations on the dataset

- Does not require fine-tuning, can be used with any LLM

- May not require as powerful of an LLM for the primary prompt

Cons:

- Cannot be used with sensitive data in the DB, basically assume users have full read only access to it

- Requires many LLM passes for good quality responses and can potentially cost a lot on expensive models

- Still requires pretty good LLM for the SQL gen + reflection.

- Pretty slow

# Usage

## Development

- `pip install -r requirements.txt` in a venv

- needs an .env file in the root directory:

    ```bash

    MYSQL_ROOT_PASSWORD=pass

    MYSQL_DATABASE=prompt_db

    DB_HOST=localhost

    DB_USERNAME=user

    DB_PASSWORD=pass

    OPENAI_API_KEY=

    ```

- Run the SQL container using `docker compose up`

- The local DB requires user-login.txt and user-pass.txt in the root directory (same one as in .env file)

- `python src/main.py` to run the dev server

## Prod

Use gunicorn with `main:app` set timeout to high and pass ENV_FILE environment variable pointing to your .env file