{"id":50405885,"url":"https://github.com/noosxe/llama-launcher","last_synced_at":"2026-05-31T01:30:44.244Z","repository":{"id":352824447,"uuid":"1211489443","full_name":"noosxe/llama-launcher","owner":"noosxe","description":"Quick launcher for running LLMs with llama-server containers","archived":false,"fork":false,"pushed_at":"2026-05-23T15:40:28.000Z","size":53,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T17:23:04.320Z","etag":null,"topics":["launcher","llama-cpp","local-llm"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/noosxe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-15T12:52:46.000Z","updated_at":"2026-05-23T15:40:25.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/noosxe/llama-launcher","commit_stats":null,"previous_names":["noosxe/llama-launcher"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/noosxe/llama-launcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noosxe%2Fllama-launcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noosxe%2Fllama-launcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noosxe%2Fllama-launcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noosxe%2Fllama-launcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/noosxe","download_url":"https://codeload.github.com/noosxe/llama-launcher/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noosxe%2Fllama-launcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33716338,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["launcher","llama-cpp","local-llm"],"created_at":"2026-05-31T01:30:44.122Z","updated_at":"2026-05-31T01:30:44.237Z","avatar_url":"https://github.com/noosxe.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🦙 Llama Launcher\n\n[![Release](https://github.com/noosxe/llama-launcher/actions/workflows/release.yml/badge.svg)](https://github.com/noosxe/llama-launcher/actions/workflows/release.yml) [![Go Reference](https://pkg.go.dev/badge/github.com/noosxe/llama-launcher/cmd/llama-launcher.svg)](https://pkg.go.dev/github.com/noosxe/llama-launcher) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n\u003e [!CAUTION]\n\u003e **Warning**: This project is fully **vibe-coded**. Proceed with appropriate levels of curiosity and caution. The auther (me) will be adding various comments throughout this readme and maybe other documents or code manually. You will probably notice them immediately because they will not align with general \"vibe\" of llms. \n\n**Llama Launcher** is a highly polished (not really!), interactive Terminal User Interface (TUI) built in Go to seamlessly orchestrate and manage `llama.cpp` Docker containers. \n\nInstead of juggling massive shell commands to spin up different local LLMs, Llama Launcher serves as a centralized dashboard. It allows you to rapidly deploy, supervise, monitor, and selectively tear down highly-configured local inference nodes!\n\n## ✨ Features\n\n- **Detached Orchestration**: Your inference instances are fully decoupled! You can launch a container inside the dashboard, press `Q` to completely exit the launcher, and your underlying `docker` workload will stay actively serving! Opening the dashboard again instantly rediscovers and re-attaches to the background logs.\n- **Hardware Monitoring**: Check real-time Global Host Stats via the interactive footer, which utilizes native Linux tools (`top`, `free`, and `nvidia-smi`) to poll accurate CPU, RAM, GPU, and VRAM utilization continuously!\n- **Catppuccin Themes**: Ships natively with full support for the gorgeous [Catppuccin](https://github.com/catppuccin/catppuccin) color standard. Customize your entire interface on the fly with a dedicated settings menu! (Disclaimer: I've tested the \"mocha\" variant only, as it is the color scheme I use in my terminal. Other variants of catppuccin may not work as intended. I'm an idiot, not a masochist. (ok, I've left the idiot not a masochist part here because gemini suggested that when I was typing. :shrug:))\n- **Dynamic Configuration**: Hot-swap configurations utilizing standard TOML tables. Expand standard pathing variables seamlessly into your volumes! (no idea what it meant by hot-swap here, not tested at all!)\n\n## ⚙️ Prerequisites\n\n- **Go 1.22+**\n- **Docker**\n- **NVIDIA Container Toolkit** (for GPU acceleration/rendering)\n\n## 🚀 Installation \u0026 Usage\n\n1. **Install directly**\n   ```bash\n   go install github.com/noosxe/llama-launcher/cmd/llama-launcher@latest\n   ```\n   *Alternatively, clone and build:*\n   ```bash\n   git clone https://github.com/noosxe/llama-launcher.git\n   cd llama-launcher\n   go build -o llama-launcher ./cmd/llama-launcher\n   ```\n3. **Set up the config**\n   Copy the example config and adjust the paths to point toward your native `.gguf` directories.\n   ```bash\n   cp config.example.toml config.toml\n   ```\n4. **Launch**\n   ```bash\n   ./llama-launcher tui\n   ```\n\n## 🎮 Interface Controls\n\n- `Up` / `Down` or `J` / `K`: Navigate models in the sidebar.\n- `Enter`: Boot or gracefully kill a model endpoint.\n- `Escape`: Toggle the central Overlay Menu to switch internal Themes or confidently exit the application.\n- `Q` or `Ctrl+C`: Gracefully sever the log listeners and exit the application instantly *(Note: Background processes stay completely unbothered!)*\n\n## 📝 Configuration (`config.toml`)\n\nLlama Launcher utilizes a global/local cascading configuration. Values defined at the top level act as defaults and can be overridden by individual model definitions.\n\n### Global Options\n\n| Option | Description |\n|--------|-------------|\n| `container_image` | Default Docker image to use (e.g., `ghcr.io/ggerganov/llama.cpp:server`) |\n| `model_dir` | The base directory on your host where models are stored. Supports `~` expansion. |\n| `port` | Default host port to map if not specified in the model. |\n| `n_predict` | Default number of tokens to predict (`-n`). |\n| `chat_template` | Default chat template name or content (`--chat-template`). |\n| `ctk` | Default KV cache key quantization (`-ctk`). |\n| `ctv` | Default KV cache value quantization (`-ctv`). |\n| `jinja` | Pass the Jinja template flag to the server (`--jinja`). Defaults to true. |\n\n### Model Options (`[[models]]`)\n\nEach entry in the `[[models]]` list defines a specific model configuration.\n\n| Option | Type | Required | Description |\n|--------|------|----------|-------------|\n| `name` | string | **Yes** | Display name in the TUI sidebar. |\n| `container_name`| string | **Yes** | Unique name for the Docker container (`--name`). |\n| `model_file` | string | No* | The GGUF filename inside `model_dir`. |\n| `model_path` | string | No* | Full path to the model file. (Alternative to `model_file`). |\n| `container_image`| string | No | Override for the global `container_image`. |\n| `model_dir` | string | No | Override for the global `model_dir`. |\n| `host_port` | int | No | The port on the host machine to bind to. |\n| `container_port`| int | No | The port inside the container (default: 8080). |\n| `gpu_layers` | int | No | Number of layers to offload to GPU (`--n-gpu-layers`). |\n| `context_size`| int | No | Context window size (`-c`). |\n| `threads` | int | No | Number of threads to use (`-t`). |\n| `batch_size` | int | No | Physical batch size (`-b`). |\n| `n_predict` | int | No | Number of tokens to predict (`-n`). |\n| `chat_template`| string | No | Chat template override (`--chat-template`). |\n| `ctk` | string | No | KV cache key quantization override (`-ctk`). |\n| `ctv` | string | No | KV cache value quantization override (`-ctv`). |\n| `jinja` | bool | No | Jinja template flag override (`--jinja`). |\n\n*\\* Note: You must provide either `model_file` (if `model_dir` is set) or `model_path` so the launcher can find the model.*\n\n### Full Example\n\n```toml\ncontainer_image = \"local/llama.cpp:server-cuda\"\nmodel_dir = \"~/models\"\nport = 8080\njinja = true\n\n[[models]]\nname = \"tiny-llama\"\nmodel_file = \"tiny-llama-1b.q4_k_m.gguf\"\ncontainer_name = \"llama-tiny\"\ngpu_layers = 32\ncontext_size = 2048\n\n[[models]]\nname = \"mistral-7b\"\nmodel_file = \"mistral-7b-instruct-v0.1.q4_k_m.gguf\"\ncontainer_name = \"llama-mistral\"\nhost_port = 8081\ngpu_layers = 33\nctk = \"q8_0\"\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoosxe%2Fllama-launcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnoosxe%2Fllama-launcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoosxe%2Fllama-launcher/lists"}