https://github.com/niansa/discord_llama

Multi-Model and multi-tasking llama Discord Bot - Mirror of: https://gitlab.com/niansa/discord_llama
https://github.com/niansa/discord_llama

ai cpp20 discord-bot llama llama2 llamacpp llm llm-inference

Last synced: 7 days ago
JSON representation

Multi-Model and multi-tasking llama Discord Bot - Mirror of: https://gitlab.com/niansa/discord_llama

• Host: GitHub
• URL: https://github.com/niansa/discord_llama
• Owner: niansa
• License: wtfpl
• Created: 2023-07-13T14:37:32.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2024-03-27T17:42:21.000Z (10 months ago)
• Last Synced: 2025-01-01T09:32:27.376Z (19 days ago)
• Topics: ai, cpp20, discord-bot, llama, llama2, llamacpp, llm, llm-inference
• Language: C++
• Homepage:
• Size: 14 MB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Discord LLaMa

## Overview

This Discord bot lets you use GPT-J and LLaMa models to both participate in your conversations and answer your questions in thread channels, in a ChatGPT-like fashion!

## Setup

On a fresh Debian 11 install, you'll need the following packages: `git` `g++` `make` `cmake` `zlib1g-dev` `libssl-dev` `libsqlite3-dev`. For better performance with large batch sizes, `libopenblas-dev` is recommended.

Then, you can clone this repository including all its submodules:

    git clone https://gitlab.com/niansa/discord_llama.git --recursive

Once the command has completed, create a build directory, configure the project and compile it:

    cd discord_llama

    mkdir build

    cd build

    cmake ..

    make -j$(nproc)

We should now download a model. Let's start with `gpt4all-j-v1.3-groovy`! I've already written an example model config for it, so let's just grab that and download the model:

    mkdir models

    cd models

    cp ../../example_models/gpt4all-j-v1.3-groovy.txt .

    wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin

    cd ..

Next, we need a configuration file. Let's just grab the example config:

    cp ../example_config.txt ./config.txt

and edit it. For reference, you can read [explained_config.txt](https://gitlab.com/niansa/discord_llama/-/blob/master/explained_config.txt).

Please note that `gpt4all-j-v1.3-groovy` is only capable of instruct mode, so you'll have to set `threads_only` to `true`, if you've downloaded just that model.

Finally, start the Discord Bot by passing the config file to the executable:

    ./discord_llama config.txt

And that's it! Feel free to play around, try different models, tweak the config file, etc... It's really easy! If you still have any questions, please write an issue or contact me on Discord: *Tuxifan#0981*.

## Credits

A huge thank you to *Nomic AI* for helping me get this project to where it is now!

Also, this project wouldn't have been possible without the effort of *Georgi Gerganov (ggerganov)*, who has writte the core of it all, the inference code itself.