https://github.com/bbc-esq/vectordb-plugin-for-koboldai

Basic vector database plugin that works wth KoboldAI. Adds documents, images, and audio files.
https://github.com/bbc-esq/vectordb-plugin-for-koboldai

ctranslate2 database koboldai koboldcpp sentence-transformers tiledb transformers vector-database vision-transformer whispers2t

Last synced: about 1 month ago
JSON representation

Basic vector database plugin that works wth KoboldAI. Adds documents, images, and audio files.

• Host: GitHub
• URL: https://github.com/bbc-esq/vectordb-plugin-for-koboldai
• Owner: BBC-Esq
• Created: 2024-07-22T01:13:15.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-08-07T19:41:52.000Z (5 months ago)
• Last Synced: 2024-08-07T22:56:53.644Z (5 months ago)
• Topics: ctranslate2, database, koboldai, koboldcpp, sentence-transformers, tiledb, transformers, vector-database, vision-transformer, whispers2t
• Language: Python
• Homepage:
• Size: 979 KB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        


  
🚀 KoboldCPP Vector Database





* 🔥 Due to time constrains and ability to test, only supported on ```Windows``` systems with an ```Nvidia GPU```.

 
REQUIREMENTS



  

1) 🐍[Python 3.11](https://www.python.org/downloads/release/python-3119/) (Pytorch is not compatible with Python 3.12 yet)

2) 📁[Git](https://git-scm.com/downloads)

3) 📁[Git Large File Storage](https://git-lfs.com/).

4) 🌐[Pandoc](https://github.com/jgm/pandoc/releases).

5) CUDA is technically "required" but the installation script installs it automatically.  You do not need to have it installed systemwide.

6) Build Tools.

   > Certain dependencies don't have pre-compiled "wheels" so you must build them with something like [Microsoft Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/) and/or [Visual Studio](https://visualstudio.microsoft.com/).  I recommend Visual Studio, but make sure to select the "Desktop development with C++" extension and check the four boxes on the right containing "SDK."


   

     EXAMPLE ERROR ON WINDOWS

     

   

   

     EXAMPLE SOLUTION ON WINDOWS

     

   

 
INSTALLATION



  

### Step 1

Download the latest "release," extract its contents, navigate to the "src" folder to run the commands below.

  * NOTE: If you clone this repository you WILL NOT get the latest release.  Instead, you will development versions of this program which may or may not be stable.

### Step 2

Navigate to the ```src``` folder, open a command prompt, and create a virtual environment:

```

python -m venv .

```

### Step 3

Activate the virtual environment:

```

.\Scripts\activate

```

### Step 4

Run setup:

```

python setup_windows.py

```

   > If you need to try reinstalling for any reason run this command to uninstall everything and try again:

```pip freeze > requirements_uninstall.txt && pip uninstall -r requirements_uninstall.txt -y && del requirements_uninstall.txt```

### Step 5

```

python gui.py

```


 
USAGE



### Start Kobold

* Navigate to KoboldCPP's github and download the appropriate .exe file or run ```python download_kobold.py``` from your virtual environment instead.

* Select any/all options you want and start the program, but make sure and uncheck the open browser option.  You can also check the "quiet" mode, which prevents the Kobold's command prompt window from printing everything in duplicate.