https://github.com/reinterpretcat/local_assist
A simple GUI for local hosted LLM (via Ollama)
https://github.com/reinterpretcat/local_assist
ai llm llm-apps ollama-gui
Last synced: 6 months ago
JSON representation
A simple GUI for local hosted LLM (via Ollama)
- Host: GitHub
- URL: https://github.com/reinterpretcat/local_assist
- Owner: reinterpretcat
- Created: 2024-12-31T11:50:49.000Z (9 months ago)
- Default Branch: master
- Last Pushed: 2025-02-12T20:55:50.000Z (8 months ago)
- Last Synced: 2025-02-12T21:25:04.132Z (8 months ago)
- Topics: ai, llm, llm-apps, ollama-gui
- Language: Python
- Homepage:
- Size: 330 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Personal AI Chat Assistant
An experimental AI Chat client built with Python/Tkinter for experimentation with local hosted LLMs (ollama). This project represents a "learn by doing" approach where various LLMs (including the ones consumed through the chat client itself) helped in the development process.
## Project Philosophy
This is an experimental project focusing on functionality over code architecture. Some interesting aspects:
- Built primarily for personal use and experimentation with local LLM models
- Code was generated with help of various LLMs
- The application was partially used to improve itself - it generated some of its own code!
- Prioritizes rapid experimentation over architectural patterns (no tests, no good design)
- My primary use cases:
- Learning AI Engineering: experimental playground for understanding capabilities of local hosted LLM on consumer hardware
- Language learning through AI conversation (e.g. with German TTS/STT configs)
- Code assistant
## Key Features
- Integration with Ollama for local LLM hosting
- Chat history with sqlite database and ability export/import chat data to json
- Text-to-Speech (TTS) for AI responses
- Speech-to-Text (STT) for voice input
- Retrieval-Augmented Generation (RAG) support for chatting about your documents (via llama-index)
- Code editor with ability to run code for some languages (python/rust)
- Customizable system prompts and LLM settings (per chat)
- Custom themes support
- Basic markdown rendering support and syntax highlighting
- Basic multi-model support (Image to Text)
## Installation
1. Install Ollama from https://ollama.com/download
2. Pull desired LLM models:
```bash
# Basic models
ollama pull llama2:13b
ollama pull mistral:7b
# Code-specific models
ollama pull qwen2.5-coder:7b
# Language learning models (German)
ollama pull cyberwald/llama-3.1-sauerkrautlm-8b-instruct:q8_0
# Multi-modal (to work with images)
ollama run llava-phi3
```
3. Install Python dependencies:
```bash
pip install -r requirements.txt
```
4. Launch the application:
```bash
python3 -m src.main
```
Note: On first run, the application will automatically download required voices and dependencies for STT/TTS functionality.
## Usage Examples
### Basic Usage
```bash
# Default configuration
python3 -m src.main
# Check CUDA availability
python3 -c 'import torch; print(torch.cuda.is_available())'
```
### Language Learning Configurations
```bash
# German learning mode
python3 -m src.main --config config-de.json
# Code assistant mode
python3 -m src.main --config config-code.json
```
## RAG (Retrieval-Augmented Generation)
The application supports chatting with your documents through RAG:
- Upload PDF, Markdown and TXT document formats
- Automatic document embedding and indexing
- Context-aware responses based on your documents
## Known Limitations
- Code structure prioritizes experimentation over maintainability
- Some features may be unstable or experimental
- UI is functional but basic
- Limited error handling in some areas
## License
This project is open source and available under the MIT License.
## Acknowledgments
- Initial inspiration from the [june](https://github.com/mezbaul-h/june) project
- Various LLMs that helped generate and improve the code