{"id":23331073,"url":"https://github.com/capsize-games/airunner","last_synced_at":"2025-12-12T02:12:58.868Z","repository":{"id":148312035,"uuid":"617136332","full_name":"Capsize-Games/airunner","owner":"Capsize-Games","description":"Privacy focused, local-first, multi-modal inference engine and agent platform for running LLMs, image generation, speech processing, and tool-based automation","archived":false,"fork":false,"pushed_at":"2025-05-13T20:49:28.000Z","size":23295,"stargazers_count":606,"open_issues_count":50,"forks_count":43,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-05-13T20:59:57.071Z","etag":null,"topics":["ai","ai-art","art","asset-generator","chatbot","deep-learning","desktop-app","image-generation","mistral","multimodal","privacy","pygame","pyside6","python","self-hosted","speech-to-text","stable-diffusion","text-to-image","text-to-speech","text-to-speech-app"],"latest_commit_sha":null,"homepage":"https://capsizegames.itch.io/ai-runner","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Capsize-Games.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"w4ffl35","patreon":"w4ffl35","ko_fi":null}},"created_at":"2023-03-21T19:14:03.000Z","updated_at":"2025-05-12T19:53:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"9b7ea487-d6d0-4a3b-a0db-b7046407a167","html_url":"https://github.com/Capsize-Games/airunner","commit_stats":null,"previous_names":[],"tags_count":224,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Capsize-Games%2Fairunner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Capsize-Games%2Fairunner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Capsize-Games%2Fairunner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Capsize-Games%2Fairunner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Capsize-Games","download_url":"https://codeload.github.com/Capsize-Games/airunner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270646,"owners_count":22042859,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-art","art","asset-generator","chatbot","deep-learning","desktop-app","image-generation","mistral","multimodal","privacy","pygame","pyside6","python","self-hosted","speech-to-text","stable-diffusion","text-to-image","text-to-speech","text-to-speech-app"],"created_at":"2024-12-20T22:30:30.491Z","updated_at":"2025-12-12T02:12:58.858Z","avatar_url":"https://github.com/Capsize-Games.png","language":"Python","readme":"# AI Runner\n\n```\nSupport development. Send crypto: 0x02030569e866e22C9991f55Db0445eeAd2d646c8\n```\n\n## Your new favorite local AI platform\n\nAI Runner is an all-in-one, offline-first desktop application, headless server, and Python library for local LLMs, TTS, STT, and image generation.\n\n\n\u003cimg src=\"./images/art_interface.png\" alt=\"AI Runner Logo\" /\u003e\n\n\n[![Discord](https://img.shields.io/discord/839511291466219541?color=5865F2\u0026logo=discord\u0026logoColor=white)](https://discord.gg/ukcgjEpc5f) ![GitHub](https://img.shields.io/github/license/Capsize-Games/airunner) [![PyPi](https://github.com/Capsize-Games/airunner/actions/workflows/pypi-dispatch.yml/badge.svg)](https://github.com/Capsize-Games/airunner/actions/workflows/pypi-dispatch.yml) ![GitHub last commit](https://img.shields.io/github/last-commit/Capsize-Games/airunner)\n\n[🐞 Report Bug](https://github.com/Capsize-Games/airunner/issues/new?template=bug_report.md) · [✨ Request Feature](https://github.com/Capsize-Games/airunner/issues/new?template=feature_request.md) · [🛡️ Report Vulnerability](https://github.com/Capsize-Games/airunner/issues/new?template=vulnerability_report.md) · [📖 Wiki](https://github.com/Capsize-Games/airunner/wiki)\n\n---\n\n## ✨ Key Features\n\n| Feature | Description |\n|---------|-------------|\n| **🗣️ Voice Chat** | Real-time conversations with LLMs using espeak or OpenVoice |\n| **🤖 Custom AI Agents** | Configurable personalities, moods, and RAG-enhanced knowledge |\n| **🎨 Visual Workflows** | Drag-and-drop LangGraph workflow builder with runtime execution |\n| **🖼️ Image Generation** | Stable Diffusion (SD 1.5, SDXL) and FLUX models with drawing tools, LoRA, inpainting, and filters |\n| **🔒 Privacy First** | Runs locally with no external APIs by default, configurable guardrails |\n| **⚡ Fast Generation** | Uses GGUF and quantization for faster inference and lower VRAM usage |\n\n### 🌍 Language Support\n\n| Language | TTS | LLM | STT | GUI |\n|----------|-----|-----|-----|-----|\n| English | ✅ | ✅ | ✅ | ✅ |\n| Japanese | ✅ | ✅ | ❌ | ✅ |\n| Spanish/French/Chinese/Korean | ✅ | ✅ | ❌ | ❌ |\n\n---\n\n## ⚙️ System Requirements\n\n| | Minimum | Recommended |\n|---|---------|-------------|\n| **OS** | Ubuntu 22.04, Windows 10 | Ubuntu 22.04 (Wayland) |\n| **CPU** | Ryzen 2700K / i7-8700K | Ryzen 5800X / i7-11700K |\n| **RAM** | 16 GB | 32 GB |\n| **GPU** | NVIDIA RTX 3060 | NVIDIA RTX 5080 |\n| **Storage** | 22 GB - 100 GB+ (actual usage varies, SSD recommended) | 100 GB+ |\n\n---\n\n## 💾 Installation\n\n### Docker (Recommended)\n\n**GUI Mode:**\n```bash\nxhost +local:docker \u0026\u0026 docker compose run --rm airunner\n```\n\n**Headless API Server:**\n```bash\ndocker compose run --rm --service-ports airunner --headless\n```\n\n\u003e **Note:** `--service-ports` is required to expose port 8080 for the API.\n\nThe headless server exposes an HTTP API on port 8080 with endpoints:\n- `GET /health` - Health check and service status\n- `POST /llm` - LLM inference\n- `POST /art` - Image generation\n\n### Manual Installation (Ubuntu/Debian)\n\n**Python 3.13+ required.** We recommend using `pyenv` and `venv`.\n\n1. **Install system dependencies:**\n   ```bash\n   sudo apt update \u0026\u0026 sudo apt install -y \\\n     build-essential cmake git curl wget \\\n     nvidia-cuda-toolkit pipewire libportaudio2 libxcb-cursor0 \\\n     espeak espeak-ng-espeak qt6-qpa-plugins qt6-wayland \\\n     mecab libmecab-dev mecab-ipadic-utf8 libxslt-dev mkcert\n   ```\n\n2. **Create data directory:**\n   ```bash\n   mkdir -p ~/.local/share/airunner\n   ```\n\n3. **Install AI Runner:**\n   ```bash\n   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128\n   pip install airunner[all_dev]\n   ```\n\n4. **Install llama-cpp-python with CUDA (Python 3.13, RTX 5080):**\n  ```bash\n  CMAKE_ARGS=\"-DGGML_CUDA=on -DGGML_CUDA_ARCHITECTURES=90\" FORCE_CMAKE=1 \\\n    pip install --no-binary=:all: --no-cache-dir \"llama-cpp-python==0.3.16\"\n  ```\n  - Uses GGML_CUDA (CUBLAS flag is deprecated).\n  - `90` matches RTX 5080 class GPUs; drop `-DGGML_CUDA_ARCHITECTURES` if you are unsure and let it auto-detect.\n  - On Python 3.12 you may instead use the prebuilt wheel: `--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121 \"llama-cpp-python==0.3.16+cu121\"`.\n\n4. **Run:**\n   ```bash\n   airunner\n   ```\n\nFor detailed instructions, see the [Installation Wiki](https://github.com/Capsize-Games/airunner/wiki/Installation-instructions).\n\n---\n\n## 🤖 Models\n\nAI Runner downloads essential TTS/STT models automatically. LLM and image models must be configured:\n\n| Category | Model | Size |\n|----------|-------|------|\n| **LLM (default)** | Llama 3.1 8B Instruct (4bit) | ~4 GB |\n| **Image** | Stable Diffusion 1.5 | ~2 GB |\n| **Image** | SDXL 1.0 | ~6 GB |\n| **Image** | FLUX.1 Dev/Schnell (GGUF) | 8-12 GB |\n| **TTS** | OpenVoice | 654 MB |\n| **STT** | Whisper Tiny | 155 MB |\n\n**LLM Providers:** Local (HuggingFace), Ollama, OpenRouter, OpenAI\n\n**Art Models:** Place your models in `~/.local/share/airunner/art/models/`\n\n---\n\n## 🛠️ CLI Commands\n\n| Command | Description |\n|---------|-------------|\n| `airunner` | Launch GUI |\n| `airunner-headless` | Start headless API server |\n| `airunner-hf-download` | Download/manage models from HuggingFace |\n| `airunner-civitai-download` | Download models from CivitAI |\n| `airunner-build-ui` | Rebuild UI from `.ui` files |\n| `airunner-tests` | Run test suite |\n| `airunner-generate-cert` | Generate SSL certificate |\n\n**Note:** To download models, use *Tools → Download Models* from the main application menu, or use `airunner-hf-download` / `airunner-civitai-download` from the command line.\n\n---\n\n## 🖥️ Headless Server\n\nAI Runner can run as a headless HTTP API server, enabling remote access to LLM, image generation, TTS, and STT capabilities. This is useful for:\n\n- Running AI services on a remote server\n- Integration with other applications via REST API\n- VS Code integration as an Ollama/OpenAI replacement\n- Automated pipelines and scripting\n\n### Quick Start\n\n```bash\n# Start with defaults (port 8080, LLM only)\nairunner-headless\n\n# Start with a specific LLM model\nairunner-headless --model /path/to/Qwen2.5-7B-Instruct-4bit\n\n# Run as Ollama replacement for VS Code (port 11434)\nairunner-headless --ollama-mode\n\n# Don't preload models - load on first request\nairunner-headless --no-preload\n```\n\n### Command Line Options\n\n| Option | Description |\n|--------|-------------|\n| `--host HOST` | Host address to bind to (default: `0.0.0.0`) |\n| `--port PORT` | Port to listen on (default: `8080`, or `11434` in ollama-mode) |\n| `--ollama-mode` | Run as Ollama replacement on port 11434 |\n| `--model, -m PATH` | Path to LLM model to load |\n| `--art-model PATH` | Path to Stable Diffusion model to load |\n| `--tts-model PATH` | Path to TTS model to load |\n| `--stt-model PATH` | Path to STT model to load |\n| `--enable-llm` | Enable LLM service |\n| `--enable-art` | Enable Stable Diffusion/art service |\n| `--enable-tts` | Enable TTS service |\n| `--enable-stt` | Enable STT service |\n| `--no-preload` | Don't preload models at startup |\n\n### Environment Variables\n\n| Variable | Description |\n|----------|-------------|\n| `AIRUNNER_LLM_MODEL_PATH` | Path to LLM model |\n| `AIRUNNER_ART_MODEL_PATH` | Path to art model |\n| `AIRUNNER_TTS_MODEL_PATH` | Path to TTS model |\n| `AIRUNNER_STT_MODEL_PATH` | Path to STT model |\n| `AIRUNNER_NO_PRELOAD` | Set to `1` to disable model preloading |\n| `AIRUNNER_LLM_ON` | Enable LLM service (`1` or `0`) |\n| `AIRUNNER_SD_ON` | Enable Stable Diffusion (`1` or `0`) |\n| `AIRUNNER_TTS_ON` | Enable TTS service (`1` or `0`) |\n| `AIRUNNER_STT_ON` | Enable STT service (`1` or `0`) |\n\n### API Endpoints\n\n#### Native AIRunner Endpoints\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/health` | Health check and service status |\n| POST | `/llm` | LLM text generation (streaming) |\n| POST | `/llm/generate` | LLM text generation |\n| POST | `/art` | Image generation |\n| POST | `/tts` | Text-to-speech |\n| POST | `/stt` | Speech-to-text |\n\n#### Ollama-Compatible Endpoints (port 11434)\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/api/tags` | List available models |\n| GET | `/api/version` | Get version info |\n| GET | `/api/ps` | List running models |\n| POST | `/api/generate` | Text generation |\n| POST | `/api/chat` | Chat completion |\n| POST | `/api/show` | Show model info |\n\n#### OpenAI-Compatible Endpoints\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| GET | `/v1/models` | List models |\n| POST | `/v1/chat/completions` | Chat completion with tool support |\n\n### Example: LLM Request\n\n```bash\ncurl -X POST http://localhost:8080/llm \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"prompt\": \"What is the capital of France?\",\n    \"stream\": true,\n    \"temperature\": 0.7,\n    \"max_tokens\": 100\n  }'\n```\n\n### Example: Image Generation (Art)\n\n```bash\n# Requires: airunner-headless --enable-art\ncurl -X POST http://localhost:8080/art \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"prompt\": \"A beautiful sunset over mountains\",\n    \"negative_prompt\": \"blurry, low quality\",\n    \"width\": 512,\n    \"height\": 512,\n    \"steps\": 20,\n    \"seed\": 42\n  }'\n# Returns: {\"images\": [\"base64_png_data...\"], \"count\": 1, \"seed\": 42}\n```\n\n### Example: Text-to-Speech (TTS)\n\n```bash\n# Requires: airunner-headless --enable-tts\ncurl -X POST http://localhost:8080/tts \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"text\": \"Hello, world!\"}'\n# Returns: {\"status\": \"queued\", \"message\": \"Text queued for speech synthesis\"}\n# Audio plays through system speakers\n```\n\n### Example: Speech-to-Text (STT)\n\n```bash\n# Requires: airunner-headless --enable-stt\n# Audio must be base64-encoded WAV (16kHz mono recommended)\ncurl -X POST http://localhost:8080/stt \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"audio\": \"UklGRi4AAABXQVZFZm10IBAAAAABAAEA...\"}'\n# Returns: {\"transcription\": \"Hello world\", \"status\": \"success\"}\n```\n\n### Example: Ollama Mode with VS Code\n\n1. Start the headless server in Ollama mode:\n   ```bash\n   airunner-headless --ollama-mode --model /path/to/your/model\n   ```\n\n2. Configure VS Code Continue extension to use `http://localhost:11434`\n\n3. The server will respond to Ollama API calls, allowing seamless integration.\n\n### Auto-Loading Models\n\nWhen `--no-preload` is used, models are automatically loaded on the first request to the corresponding endpoint. This is useful for:\n\n- Reducing startup time\n- Running multiple services without loading all models upfront\n- Memory-constrained environments\n\n---\n\n## 📦 Model Management\n\n### Download Models\n\n```bash\n# List available models\nairunner-hf-download\n\n# List only LLM models\nairunner-hf-download list --type llm\n\n# Download a model (GGUF by default)\nairunner-hf-download qwen3-8b\n\n# Download full safetensors version\nairunner-hf-download --full qwen3-8b\n\n# Download any HuggingFace model\nairunner-hf-download Qwen/Qwen3-8B\n\n# List downloaded models\nairunner-hf-download --downloaded\n```\n\n### Delete Models\n\n```bash\n# Delete a model (with confirmation)\nairunner-hf-download --delete Qwen3-8B\n\n# Delete without confirmation (for scripts)\nairunner-hf-download --delete Qwen3-8B --force\n```\n\n### Download from CivitAI\n\n```bash\n# Download a model from CivitAI URL\nairunner-civitai-download https://civitai.com/models/995002/70s-sci-fi-movie\n\n# Download a specific version\nairunner-civitai-download https://civitai.com/models/995002?modelVersionId=1880417\n\n# Download to a custom directory\nairunner-civitai-download \u003curl\u003e --output-dir /path/to/models\n\n# Use API key for authentication (for gated models)\nairunner-civitai-download \u003curl\u003e --api-key your_api_key\n\n# Or set CIVITAI_API_KEY environment variable\nexport CIVITAI_API_KEY=your_api_key\nairunner-civitai-download \u003curl\u003e\n```\n\n---\n\n## 🔒 HTTPS Configuration\n\nAI Runner's local server uses HTTPS by default. Certificates are auto-generated in `~/.local/share/airunner/certs/`.\n\nFor browser-trusted certificates, install [mkcert](https://github.com/FiloSottile/mkcert):\n```bash\nsudo apt install libnss3-tools\nmkcert -install\n```\n\n---\n\n## ⚖️ Colorado AI Act Notice\n\n**Effective February 1, 2026**, the [Colorado AI Act (SB 24-205)](https://leg.colorado.gov/bills/sb24-205) regulates high-risk AI systems.\n\n**Your Responsibility:** If you use AI Runner for decisions with legal or significant effects on individuals (employment screening, loan eligibility, insurance, housing), you may be classified as a **deployer of a high-risk AI system** and must:\n- Implement a risk management policy\n- Complete impact assessments\n- Provide consumer notice and appeal mechanisms\n- Report algorithmic discrimination to the Colorado Attorney General\n\n**AI Runner's Design:** AI Runner is designed with privacy as a core principle—it runs entirely locally with no external data transmission by default. However, certain optional features connect to external services:\n\n- **Model Downloads:** Connecting to HuggingFace or CivitAI to download models\n- **Web Search / Deep Research:** Search queries sent to DuckDuckGo; web pages scraped for research\n- **Weather Prompt:** Location coordinates sent to Open-Meteo API if enabled\n- **External LLM Providers:** Prompts sent to OpenRouter or OpenAI if configured\n\n**We recommend using a VPN** when using features that connect to external services. See our full [Privacy Policy](src/airunner/components/downloader/gui/windows/setup_wizard/user_agreement/privacy_policy.md) for details.\n\n---\n\n## 🧪 Testing\n\n```bash\n# Run headless-safe tests\npytest src/airunner/utils/tests/\n\n# Run display-required tests (Qt/GUI)\nxvfb-run -a pytest src/airunner/utils/tests/xvfb_required/\n```\n\n---\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) and the [Development Wiki](https://github.com/Capsize-Games/airunner/wiki/Development).\n\n## Documentation\n\n- [Wiki](https://github.com/Capsize-Games/airunner/wiki)\n- [API Service Layer](src/airunner/components/application/api/README.md)\n- [ORM Models](src/airunner/components/data/models/README.md)\n\n---\n\n\u003ca href=\"https://airunner.org\"\u003e\n   \u003cimg src=\"https://airunner.org/logo.png\" alt=\"AI Runner Logo\" width=\"100\"/\u003e\n\u003c/a\u003e\n\n","funding_links":["https://github.com/sponsors/w4ffl35","https://patreon.com/w4ffl35"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcapsize-games%2Fairunner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcapsize-games%2Fairunner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcapsize-games%2Fairunner/lists"}