https://github.com/jwest951227/ragchatbot

This project is a dynamic AI chatbot that can be trained from various sources, such as PDFs, documents, websites, and YouTube videos.
https://github.com/jwest951227/ragchatbot

celery django faiss langchain pinecone postgresql python rag

Last synced: about 12 hours ago
JSON representation

This project is a dynamic AI chatbot that can be trained from various sources, such as PDFs, documents, websites, and YouTube videos.

• Host: GitHub
• URL: https://github.com/jwest951227/ragchatbot
• Owner: jwest951227
• Created: 2024-04-29T07:23:45.000Z (7 months ago)
• Default Branch: master
• Last Pushed: 2024-05-31T17:14:27.000Z (6 months ago)
• Last Synced: 2024-11-16T17:03:43.498Z (about 13 hours ago)
• Topics: celery, django, faiss, langchain, pinecone, postgresql, python, rag
• Language: Python
• Homepage:
• Size: 29.3 KB
• Stars: 34
• Watchers: 2
• Forks: 6
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Dynamic AI Chatbot with Custom Training Sources

## Customizable-gpt-chatbot

This project is a dynamic AI chatbot that can be trained from various sources, such as PDFs, documents, websites, and YouTube videos. It uses a user system with social authentication through Google, and the Django REST framework for its backend. The chatbot leverages OpenAI's GPT-3.5 language model to conduct conversations and is designed for scalability and ease of use.

## Features

- Train chatbot from multiple sources (PDFs, documents, websites, YouTube videos)

- User system with social authentication through Google

- Connect with OpenAI GPT-3.5 language model for conversation

- Use Pinecone and FAISS for vector indexing

- Employ OpenAI's text-embedding-ada-002 for text embedding

- Python Langchain library for file processing and text conversion

- Scalable architecture with separate settings for local, staging, and production environments

- Dynamic site settings for title and prompt updates

- Multilingual support

- PostgreSQL database support

- Celery task scheduler with Redis and AWS SQS options

- AWS S3 bucket support for scalable hosting

- Easy deployment on Heroku or AWS

## Technologies

- Language: Python

- Framework: Django REST Framework

- Database: PostgreSQL

### Major Libraries:

- Celery

- Langchain 

- OpenAI

- Pinecone

- FAISS

## Requirements

- Python 3.8 or above

- Django 4.1 or above

- Pinecone API Key

- API key from OpenAI

- Redis or AWS SQS

- PostgreSQL database

## Future Scope

- Integration with more third-party services for authentication

- Support for additional file formats and media types for chatbot training

- Improved context-awareness in conversations

- Enhanced multilingual support with automatic language detection

- Integration with popular messaging platforms and chat applications

## How to run

- Clone the repository. `git clone https://github.com/libraiger/ragChatbot`

- Install the required packages by running `pip install -r requirements.txt`

- Run celery `celery -A config worker --loglevel=info`

- Run the command `python manage.py runserver`

- Open `http://127.0.0.1:8000/` in your browser

In linux and mac need to install 'sudo apt install python3-dev -y`

1. Make sure that you have the development libraries for libcurl installed on your system. You can install them by running the following command: `sudo apt-get install libcurl4-openssl-dev gcc libssl-dev -y`

2. Make sure that you have the latest version of pip and setuptools installed by running the following command: `pip install --upgrade pip setuptools`

3. `pip install pycurl`