Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/shinomakoi/ai-messenger
A QT GUI for large language models
https://github.com/shinomakoi/ai-messenger
exllama llama llamacpp qt6
Last synced: 12 days ago
JSON representation
A QT GUI for large language models
- Host: GitHub
- URL: https://github.com/shinomakoi/ai-messenger
- Owner: shinomakoi
- License: apache-2.0
- Created: 2023-11-15T20:44:28.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-27T17:54:34.000Z (11 months ago)
- Last Synced: 2024-02-15T03:34:57.355Z (9 months ago)
- Topics: exllama, llama, llamacpp, qt6
- Language: Python
- Homepage:
- Size: 231 KB
- Stars: 14
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# AI-Messenger
QT GUI for large language models, for Windows, Linux and Mac.
Features:
- LLama.cpp and Exllama V2 as backends
- LLaVA multimodal support
- Chat, completion, notebook modes
- Model parameter presets
- Instruct presets + V2 Tavern character cards
- Save chat sessions
Uses LLaMA.cpp and ExLLaMA V2 as backends.
**Installation:**
First make sure Python (3.10+ recommended) and GIT are installed. Then:
```
git clone https://github.com/shinomakoi/AI-Messenger
cd AI-Messenger
```
Optionally create a virtual environment (recommended)
```
python -m venv .venv
source ./.venv/bin/activate ### For Linux
.\.venv\Scripts\activate ## For Windows
```
```
pip install -r requirements.txt
```
**llama.cpp**
To use the LLaMA.cpp backend run the server in the LLaMA.cpp directory, e.g.:
```
.\server -m "models/model.gguf" -t 6 -ngl 18 -c 4096
```
**ExLLaMA V2**
To use the ExLLaMa backend run ```python examples/ws-server.py --model "model_path"``` in the ExLLaMA directory
**Usage:**
To launch the app, use:
```
python main.py
```
**Character cards**
Place V2 character cards (such as from Chub.ai) in the presets/Cards directory
Uses https://github.com/UN-GCPDS/qt-material for themes