https://github.com/mcaimi/llamachat

This repository has been moved to https://codeberg.org/mcaimi/llamachat. Please visit the new location for the latest updates.
https://github.com/mcaimi/llamachat

demo-app llama-stack openai python3 streamlit wip

Last synced: about 2 months ago
JSON representation

This repository has been moved to https://codeberg.org/mcaimi/llamachat. Please visit the new location for the latest updates.

• Host: GitHub
• URL: https://github.com/mcaimi/llamachat
• Owner: mcaimi
• License: gpl-3.0
• Created: 2025-06-16T16:23:14.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2026-04-03T14:36:52.000Z (3 months ago)
• Last Synced: 2026-04-03T18:33:23.674Z (3 months ago)
• Topics: demo-app, llama-stack, openai, python3, streamlit, wip
• Language: Python
• Homepage: https://codeberg.org/mcaimi/llamachat
• Size: 3.9 MB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# LlamaChat

[![Docker Repository on Quay](https://quay.io/repository/marcocaimi/llamachat/status "Docker Repository on Quay")](https://quay.io/repository/marcocaimi/llamachat)

**LlamaChat** is a web application that allows users to chat with Large Language Models (LLMs). It provides flexibility to interact with LLMs either locally (on-prem) using technologies like **vLLM** or **Ollama** or through remote **OpenAI-compatible endpoints**.

With LlamaChat, users can easily chat with powerful LLMs both in private environments (for enhanced privacy and control) and using cloud-based APIs for scalability.

## Features

- **Unified Chat**: Interact with LLMs hosted on-premises or remotely using **llama-stack**

- **User-Friendly Interface**: Simple web interface to initiate and maintain conversations with LLMs.

- **API Key Support**: Securely manage and switch between different API keys (for remote services).

- **Multiple Models**: Support for multiple LLM models such as GPT-based models, Ollama models, and others.

- **RAG Support**: Chat with your documents, using a vector database as embeddings backend

- **Safety Shields**: Add safety guardrails to user prompts

- **Agents Support**: Use tools to enhance the LLM capabilities

## Screenshot

![LlamaChat Screenshot](assets/screenshot.png)

## Installation

### Prerequisites

- Python 3.12+

- Streamlit

- An Ollama/vLLM instance or a public OpenAI-Compatible API endpoint.

- Llama-Stack instance running on-prem or in the cloud.

## Issues

On MacOS 26, there could be problems during importing `torchcodec` into the application due to missing DYLD_LIBRARY_PATH settings.

Project needs `ffmpeg@7`, so a workaround could be:

```

$ brew install ffmpeg@7

$ DYLD_LIBRARY_PATH="/opt/homebrew/opt/ffmpeg@7/lib:$DYLD_LIBRARY_PATH" uv run streamlit run main.py

```

```