https://github.com/tatertotterson/tater
Tater is a local-first AI platform with built-in LLMs, vision, voice, memory, automation, integrations, and ESPHome voice satellite support.
https://github.com/tatertotterson/tater
ai ai-assistant discord-bot home-assistant homekit irc-bot local-llm matrix-bot
Last synced: about 13 hours ago
JSON representation
Tater is a local-first AI platform with built-in LLMs, vision, voice, memory, automation, integrations, and ESPHome voice satellite support.
- Host: GitHub
- URL: https://github.com/tatertotterson/tater
- Owner: TaterTotterson
- License: agpl-3.0
- Created: 2025-09-23T13:25:04.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2026-05-30T21:46:47.000Z (4 days ago)
- Last Synced: 2026-05-30T23:20:44.153Z (4 days ago)
- Topics: ai, ai-assistant, discord-bot, home-assistant, homekit, irc-bot, local-llm, matrix-bot
- Language: Python
- Homepage: https://taterassistant.com
- Size: 44.1 MB
- Stars: 32
- Watchers: 4
- Forks: 4
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
taterassistant.com
**Tater** is a local-first AI platform with built-in LLM, vision, voice, memory, automation, and integration systems. **Hydra** handles reasoning, orchestration, and tool use, while Tater can run local models through **llama.cpp**, **Hugging Face Transformers**, and **MLX**, or connect to remote providers through OpenAI-compatible APIs. It includes a built-in voice system that talks directly to ESPHome devices like **VoicePE**, **Sat1**, **S3Box**, and **ReSpeaker XVF3800**, a WebUI for setup, configuration, chats, model management, and system monitoring, plus integrations across **Discord**, **Home Assistant**, **HomeKit**, **IRC**, **macOS**, **Matrix**, **Meshtastic**, **Telegram**, and even the **OG Xbox via XBMC4Xbox**.
---
## 🧩 Tater Architecture
Tater is built around a modular system:
- **Cores** → core systems that extend Tater's capabilities
- **Portals** → integrations with platforms like Discord, Home Assistant, and more
- **Verbas** → AI-driven tools and actions Tater can perform
- **Integrations** → modular provider packages for devices, services, search providers, and external APIs
These catalogs, versions, metadata, and update paths are managed through **Tater Shop**:
👉 **https://github.com/TaterTotterson/Tater_Shop**
Integration packages are maintained here:
👉 **[TaterTotterson/Tater_Integrations](https://github.com/TaterTotterson/Tater_Integrations)**
---
## Supporting Apps
Some Portals are paired with companion repos/apps that complete the end-user integration:
| App / Repo | Purpose |
| --- | --- |
| [HA Add-ons](https://github.com/TaterTotterson/hassio-addons-tater) | Home Assistant add-on repository for running Tater directly inside HAOS/Supervised setups. |
| [HomeKit Shortcuts](https://taterassistant.com/portals/homekit.html) | Shortcut guide for Siri -> HomeKit bridge -> Tater workflows. |
| [Meshtastic Bridge](https://github.com/TaterTotterson/tater_meshtastic_bridge) | Host-side BLE bridge service for connecting Tater to Meshtastic radios over a simple local API. |
| [microWakeWords](https://github.com/TaterTotterson/microWakeWords) | Tater VoicePE, Satellite1, ReSpeaker, and related ESPHome firmware plus microWakeWord model assets. |
| [microWakeWord Trainer - Apple Silicon](https://github.com/TaterTotterson/microWakeWord-Trainer-AppleSilicon) | Apple Silicon trainer for creating custom microWakeWord models. |
| [microWakeWord Trainer - NVIDIA Docker](https://github.com/TaterTotterson/microWakeWord-Trainer-Nvidia-Docker) | NVIDIA Docker trainer for creating custom microWakeWord models with GPU acceleration. |
| [NanoWakeWord Trainer](https://github.com/TaterTotterson/nanoWakeWord-Trainer) | Trainer for custom NanoWakeWord models used by Tater's local or standalone NanoWakeWord server. |
| [openWakeWord Trainer](https://github.com/TaterTotterson/openWakeWord-Trainer) | Trainer for custom openWakeWord models used by Tater's local or standalone openWakeWord server. |
| [Tater MacOS](https://github.com/TaterTotterson/Tater-MacOS) | Menu bar companion app and bridge client for desktop chat, quick actions, and uploads. |
| [Reachy Mini Voice Satellite](https://huggingface.co/spaces/TaterTotterson/tater_voice_sat) | Reachy Mini robot app that turns Reachy Mini into a voice satellite for Tater or Home Assistant. |
| [Reachy Mini Tater Standalone](https://huggingface.co/spaces/TaterTotterson/tater_reachy_standalone) | Reachy Mini robot app that can run the full Tater app/stack directly on Reachy. |
| [Tater NWW Server](https://github.com/TaterTotterson/Tater-NWW-Server) | Standalone NanoWakeWord WebSocket server for Tater satellites using remote NanoWakeWord wake detection. |
| [Tater OWW Server](https://github.com/TaterTotterson/Tater-OWW-Server) | Standalone openWakeWord WebSocket server for Tater satellites that need remote wake detection outside the main Tater app. |
| [Tater S3Box Display](https://github.com/TaterTotterson/Tater-S3Box-Display) | ESP32-S3-BOX display firmware for Tater voice and dashboard-style device experiences. |
| [XBMC4Xbox Skin](https://github.com/TaterTotterson/skin.cortana.tater-xbmc) | OG Xbox/XBMC4Xbox skin and script integration for on-console Tater access. |
---
# Installation
> **Note**:
> - Tater can run any compatible local or OpenAI-compatible model. If you use a thinking model, disable thinking for best Hydra/tool behavior. Tater's built-in local providers try to suppress thinking automatically where supported.
## Local Installation
### Prerequisites
- Python 3.11
- A local OpenAI-compatible LLM runtime (such as **Ollama**, **LocalAI**, **LM Studio**, or **Lemonade**) or Tater's built-in **Hugging Face Transformers**, **llama.cpp GGUF**, or **MLX LM** providers
- Docker is optional.
### Set Up Tater
1. **Clone the Repository**
```bash
git clone https://github.com/TaterTotterson/Tater.git
```
2. **Navigate to the Project Directory**
```bash
cd Tater
```
3. **Run Tater Setup**
Use the interactive setup menu to choose the right local runtime profile:
```bash
sh setup_tater.sh
```
The setup menu creates `.venv`, installs Tater's Python dependencies, and writes the selected runtime profile to `.runtime/tater_profile.env`.
Available local profiles:
- **CPU**: safe default for most local Linux installs and generic ARM hosts.
- **macOS Apple Silicon**: native Mac setup with Apple Metal/MPS for PyTorch-backed SpeechBrain and Kokoro when available, plus MLX Whisper for local STT.
- **NVIDIA desktop/server**: native amd64 CUDA setup for RTX/GTX machines.
- **AMD ROCm / Strix Halo**: native Linux setup for ROCm-capable Radeon and Ryzen AI Max / Strix Halo systems.
- **Jetson**: native ARM64 setup that uses JetPack/system AI packages and CUDA when compatible Python runtimes are installed.
- **Jetson Thor**: native ARM64 setup for Thor / JetPack 7 systems and CUDA 13-compatible JetPack runtimes.
Non-interactive setup is also available:
```bash
sh setup_tater.sh cpu
sh setup_tater.sh macos
sh setup_tater.sh nvidia
sh setup_tater.sh rocm
sh setup_tater.sh jetson
sh setup_tater.sh thor
```
### Local Voice Acceleration Notes
The setup profile only prepares the runtime. Actual voice model choices are managed in TaterOS under **Settings -> Models** and **Settings -> ESPHome -> Voice Pipeline**.
macOS Apple Silicon:
- The macOS profile writes `PYTORCH_ENABLE_MPS_FALLBACK=1` so PyTorch can fall back to CPU for unsupported MPS operations.
- It attempts to install `mlx-whisper` and the official PyTorch `kokoro` package.
- It installs the Metal `llama-cpp-python` wheel on Apple Silicon when available; MLX LM remains the preferred Apple-native local LLM provider.
- Select **Settings -> Models -> STT Backend -> MLX Whisper** for Apple-native Whisper STT.
- MLX Whisper defaults to `mlx-community/whisper-base.en-mlx`; set `TATER_MLX_WHISPER_MODEL` to use another MLX Whisper model.
- Kokoro automatically uses the PyTorch engine on Apple Metal/MPS when available. Set `TATER_KOKORO_ENGINE=onnx` to force the existing ONNX path or `TATER_KOKORO_ENGINE=torch` to force PyTorch.
If native macOS dependency builds fail, install these Homebrew packages and rerun setup:
```bash
brew install ffmpeg libolm pkg-config
```
NVIDIA desktop/server:
- The `nvidia` profile installs CUDA PyTorch wheels, CUDA/cuDNN runtime packages, a CUDA-enabled `llama-cpp-python`, and the GPU ONNX Runtime build.
- `llama-cpp-python` defaults to the upstream CUDA 12.4 wheel index (`TATER_LLAMA_CPP_CUDA_WHEEL=cu124`) because that index carries current published CUDA wheels; set `TATER_LLAMA_CPP_CUDA_WHEEL=source` to compile locally with `CMAKE_ARGS="-DGGML_CUDA=on"` instead.
- In TaterOS, use **Settings -> Models -> Voice Acceleration** to select Auto, CPU, NVIDIA CUDA, AMD ROCm, or Apple Metal/MPS where supported.
- Faster Whisper compute type defaults to Auto. Auto uses `float16` on newer CUDA GPUs and switches to `int8` on older CUDA cards such as Pascal / GTX 10-series, where `float16` can fail.
- To override Faster Whisper compute type, use **Settings -> ESPHome -> Voice Pipeline -> Speech Recognition -> Faster Whisper Compute Type** or set `TATER_FASTER_WHISPER_COMPUTE_TYPE` to `auto`, `int8`, `float32`, `float16`, `int8_float32`, or `int8_float16`.
- To restrict which GPUs native Tater can see, start it with `CUDA_VISIBLE_DEVICES=0 sh run_ui.sh` or use a GPU UUID.
AMD ROCm / Strix Halo:
- The `rocm` profile installs PyTorch from the ROCm wheel index, then installs Tater dependencies and the official PyTorch Kokoro package.
- Tater keeps the ROCm PyTorch wheel in place when installing dependencies so Hugging Face Transformers can use ROCm through PyTorch when the device is supported.
- AMD ROCm support is Linux-only and depends on the ROCm runtime installed for the GPU/APU.
- Tater uses ROCm for PyTorch-backed models such as Kokoro Torch and SpeechBrain Speaker ID / Emotion ID. PyTorch ROCm exposes devices through the `cuda` API internally, but Tater labels it separately as AMD ROCm in settings and logs.
- llama.cpp ROCm/hipBLAS is not installed automatically; build it locally with `TATER_LLAMA_CPP_CMAKE_ARGS="-DGGML_HIPBLAS=on"` if you want GGUF GPU offload on AMD.
- Faster Whisper still falls back to CPU unless its CTranslate2 backend reports CUDA support; ROCm acceleration is not assumed for Faster Whisper.
- Strix Halo may require newer AMD ROCm wheels than the default PyTorch index. Override the PyTorch ROCm wheel source with `TATER_ROCM_PYTORCH_INDEX_URL` before running setup if needed.
Jetson and Thor:
- The `jetson` and `thor` profiles create a venv with `--system-site-packages` so NVIDIA JetPack-provided Python AI packages can be reused.
- Setup intentionally avoids replacing JetPack PyTorch with generic pip wheels.
- Hugging Face Transformers can use JetPack CUDA when the system PyTorch install exposes CUDA. llama.cpp CUDA on Jetson/Thor usually requires a local source build and is not forced by setup.
General voice notes:
- Tater warms selected local STT/TTS models at startup and after saving voice model settings. Set `TATER_SPEECH_WARMUP_ON_STARTUP=false` to disable startup warmup.
- Kokoro and Pocket TTS output are boosted slightly by default for clearer satellite playback. Tune them in Settings -> Models -> Speech -> TTS, or override local runs with `TATER_KOKORO_OUTPUT_GAIN` / `TATER_POCKET_TTS_OUTPUT_GAIN`; both default to `1.5`.
- Voice activity detection defaults to Silero VAD. Low-power hosts can switch the Voice Pipeline VAD backend to WebRTC, which uses `webrtcvad-wheels`.
- If Speaker ID or Emotion ID is enabled, SpeechBrain can use CUDA or MPS when supported, with CPU fallback.
### Run the Web UI
Start the TaterOS backend/frontend:
```bash
sh run_ui.sh
```
If `.venv` exists, `run_ui.sh` uses it automatically. It also loads `.runtime/tater_profile.env` when present.
The launcher listens on `0.0.0.0:8501` by default. To change it, set `HTMLUI_PORT`:
```bash
HTMLUI_PORT=8601 sh run_ui.sh
```
Then open:
```text
http://127.0.0.1:8501
```
Once the WebUI is up, continue to **Post-Install Setup** below.
## Docker Installation
### 1. Pull the Image
Pull the prebuilt image with the following command:
```bash
docker pull ghcr.io/tatertotterson/tater:latest
```
### 2. Run Container
Recommended Docker networking:
- Use `--network host` so Tater shares the host network directly.
- This avoids managing a growing list of `-p` mappings for WebUI, voice, and other runtime surfaces.
- With host networking, Tater listens on the host directly, so you do not need to publish Tater ports manually.
- To change the WebUI port, set `HTMLUI_PORT`, for example `-e HTMLUI_PORT=8601`.
- If you are not using host networking, publish the same container port, for example `-p 8601:8601`.
Important for Docker persistence:
- Add a path mapping for `/app/agent_lab` (container) -> `/mnt/user/appdata/tater/agent_lab` (host example).
- Without this mapping, data in `/agent_lab` (logs/downloads/documents/workspace) can be lost on container rebuilds/updates.
- Add a path mapping for `/app/.runtime` (container) -> `/mnt/user/appdata/tater/runtime` (host example).
- Without this mapping, local runtime settings can be lost on container rebuilds/updates.
---
Example: Docker setup
```
docker run -d --name tater_webui \
--network host \
-e TZ=America/Chicago \
-e HTMLUI_PORT=8501 \
-v /etc/localtime:/etc/localtime:ro \
-v /etc/timezone:/etc/timezone:ro \
-v /agent_lab:/app/agent_lab \
-v /tater_runtime:/app/.runtime \
ghcr.io/tatertotterson/tater:latest
```
### NVIDIA Docker
The NVIDIA image is amd64-only. Use the default `latest` image for CPU-first installs and ARM hosts.
The NVIDIA image uses CUDA 12.8 PyTorch wheels, CUDA/cuDNN runtime packages, GPU ONNX Runtime, and the upstream CUDA `llama-cpp-python` wheel for RTX 30, 40, and 50 series cards. Voice model tuning, Faster Whisper compute type, warmup, VAD, SpeechBrain acceleration, and llama.cpp GGUF offload use the same TaterOS settings described in **Local Voice Acceleration Notes**.
Host requirements:
- Install the NVIDIA driver.
- Install NVIDIA Container Toolkit before starting the compose override.
- The CUDA llama.cpp wheel needs `libcuda.so.1`, which is supplied by the host driver at container runtime. If diagnostics mention `libcuda.so.1`, the image built correctly but the container was not started with NVIDIA GPU access.
Optional NVIDIA GPU build for Faster Whisper STT plus Kokoro TTS:
```
docker compose -f docker-compose.yml -f docker-compose.nvidia.yml up --build
```
Prebuilt NVIDIA image:
```bash
docker pull ghcr.io/tatertotterson/tater:nvidia
```
To restrict which GPUs Tater can see in the NVIDIA compose setup, set `NVIDIA_VISIBLE_DEVICES` before launching, for example `NVIDIA_VISIBLE_DEVICES=0` or a GPU UUID. Inside the container, CUDA device `0` maps to the first visible GPU.
Build and push the NVIDIA image:
```bash
docker buildx build \
--platform linux/amd64 \
-f Dockerfile.nvidia \
-t ghcr.io/tatertotterson/tater:nvidia \
--push .
```
### 3. Access the Web UI
Once the container is running with host networking, open your browser and navigate to:
- [http://localhost:8501](http://localhost:8501) from the same machine
- `http://:8501` from another device on your network
If you changed `HTMLUI_PORT`, use that port in the URL.
Once the WebUI is up, continue to **Post-Install Setup** below.
---
## Unraid Installation
Tater is available in the **Unraid Community Apps** store.
You can install **Tater** directly from the Unraid App Store with a one-click template.
Unraid note:
- Add container path mappings for `/app/agent_lab` and `/app/.runtime` to persistent shares, for example `/mnt/user/appdata/tater/agent_lab` and `/mnt/user/appdata/tater/runtime`.
- Also set `TZ` and map `/etc/localtime` plus `/etc/timezone` if you want local time inside the container.
Once the Unraid containers are installed and running, continue to **Post-Install Setup** below.
---
## Home Assistant Installation
A dedicated Home Assistant add-on repository is available here:
https://github.com/TaterTotterson/hassio-addons-tater
Click the button below to add the repository to Home Assistant:
[](
https://my.home-assistant.io/redirect/supervisor_add_addon_repository/?repository_url=https://github.com/TaterTotterson/hassio-addons-tater
)
Once added, the **Tater AI Assistant** add-on will appear in the Home Assistant Add-on Store.
Install order:
1. Install Tater AI Assistant.
2. Configure your LLM settings in the Tater add-on.
3. Start Tater.
Once the add-ons are running, continue to **Post-Install Setup** below.
---
## Post-Install Setup
After Tater is running, open TaterOS and finish the first-run setup:
1. Configure your base model in **Settings -> Models -> LLM / Vision**:
- choose `OpenAI-Compatible API` for Ollama, LM Studio, LocalAI, Lemonade, vLLM, or a hosted compatible API
- choose `Hugging Face Transformers` to load a local model directly inside Tater
- choose `llama.cpp GGUF` to load a GGUF model through llama-cpp-python
- choose `MLX LM (Apple Silicon)` to load an MLX model directly on an Apple Silicon Mac
- for built-in local providers, download models from the Hugging Face mini-tab first, then select the downloaded model from the Settings mini-tab
- for OpenAI-compatible providers, set the endpoint host/port and model name
2. Optional:
- add more Base servers for round-robin regular AI calls
- enable `Beast Mode` and set per-head model settings for Chat/Astraeus/Thanatos/Minos/Hermes
Hydra model settings are saved by TaterOS and used at runtime. Base, Spudex, Beast Mode routing, and Vision can each use the selected built-in local providers or OpenAI-compatible providers.
### Local Models
- Download local Hugging Face Transformers, llama.cpp GGUF, or MLX models from the Hugging Face mini-tab first, then select them from Settings.
- Model caches live under `agent_lab/models/llm/` by default:
- `huggingface` for Transformers
- `llama-cpp` for GGUF models and matching `mmproj*.gguf` vision projectors
- `mlx` for MLX text and vision models
- The Hugging Face browser uses the token saved in **Integration Manager -> Hugging Face** for private/gated models and better Hub rate limits.
- llama.cpp uses GPU offload by default when the installed build supports it. Set `TATER_LLAMA_CPP_N_GPU_LAYERS=0` for CPU-only.
- MLX is intended for Apple Silicon Macs. Use llama.cpp GGUF on Linux, Raspberry Pi, NVIDIA, AMD/ROCm, Jetson, or other non-Apple-Silicon devices.
### Vision
- Vision can use an OpenAI-compatible API, the loaded Base model, or a dedicated local vision model.
- If Base is already loaded and vision-capable, Tater reuses it instead of loading the same model twice.
- Dedicated vision models are managed separately from Base.
### Advanced Notes
- Local context length is configured in **Settings -> Models -> LLM / Vision**.
- Thinking suppression is enabled by default for local providers when supported.
- `run_ui.sh` starts Uvicorn with `--no-access-log` to suppress per-request log spam.