https://github.com/itsjrsa/vupai
Voice control for your tmux AI agent panes. On-device speech Parakeet/MLX), macOS, no cloud or API keys.
https://github.com/itsjrsa/vupai
ai-agents claude-code codex macos mlx parakeet speech-to-text text-to-speech tmux voice-control
Last synced: 6 days ago
JSON representation
Voice control for your tmux AI agent panes. On-device speech Parakeet/MLX), macOS, no cloud or API keys.
- Host: GitHub
- URL: https://github.com/itsjrsa/vupai
- Owner: itsjrsa
- License: mit
- Created: 2026-06-19T16:12:39.000Z (21 days ago)
- Default Branch: master
- Last Pushed: 2026-07-03T16:01:19.000Z (7 days ago)
- Last Synced: 2026-07-03T16:04:43.291Z (7 days ago)
- Topics: ai-agents, claude-code, codex, macos, mlx, parakeet, speech-to-text, text-to-speech, tmux, voice-control
- Language: Python
- Homepage:
- Size: 1.14 MB
- Stars: 3
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
- Agents: AGENTS.md
Awesome Lists containing this project
README
Voice UI for AI panes: push-to-talk voice control for your tmux agent panes, on macOS, with on-device speech.
*vupai* (say "voo-pie") is a **V**oice **U**I for your AI **pa**nes.
Hold a key, speak, and what you say is typed into the right tmux pane: the one
you're looking at, or an agent you call by name (*"atlas, run the tests"*).
Speech-to-text runs on-device with NVIDIA Parakeet (via Apple MLX): no cloud,
no API keys.
Built for a tmux-centric workflow where you keep several coding agents and
shells open at once and want to drive them by voice without reaching for the
mouse. New panes launch an agent by default (`claude` out of the box) and should
work with other agentic coding tools (Codex, Gemini, …), though testing so far
has focused on Claude Code.
**📚 Full documentation: [vupai.dev](https://vupai.dev/)** ·
🎬 [demo clips](https://vupai.dev/voice-commands/#see-it-in-action)
## Install
vupai is **macOS Apple-Silicon only** (it depends on Apple MLX for on-device
speech). It needs `tmux`, `sox`, and [`uv`](https://docs.astral.sh/uv/):
```bash
brew install tmux sox uv
uv tool install vupai
vupai setup # one-time bootstrap: mic, hotkeys, model download, macOS permissions
```
The Parakeet model (~2 GB) downloads automatically on first use. Full details
(requirements, from-source install, permissions) are in the
[install](https://vupai.dev/install/) and
[setup](https://vupai.dev/setup/) docs.
## Quickstart
```bash
vupai # attach-or-create the session named after the cwd
```
Then hold a push-to-talk key and speak:
- **Right-Option** (dictation): what you say is typed into the focused pane.
Start with a pane's name (*"atlas, run the tests"*) to address it anywhere.
- **Right-Command** (system): voice commands that act on the panes instead of
typing: *"create 3 panes"*, *"focus atlas"*, *"board"*, *"note: fix the retry
logic"*, *"everyone, pull main"*.
It's still tmux: every normal binding keeps working, and vupai runs on its own
tmux server so your existing setup is untouched.
## Learn more
The [documentation site](https://vupai.dev/) covers everything:
- [Usage](https://vupai.dev/usage/) · [Voice commands](https://vupai.dev/voice-commands/) · [Silent mode (no mic)](https://vupai.dev/silent-mode/)
- Multi-agent supervision: [board](https://vupai.dev/board/) · [activity ledger](https://vupai.dev/activity-ledger/) · [review](https://vupai.dev/review/) · [task pile](https://vupai.dev/tasks/) · [squad](https://vupai.dev/squad/)
- Reference: [configuration](https://vupai.dev/configuration/) · [SSH hosts](https://vupai.dev/remote-hosts/) · [tmux tips](https://vupai.dev/tmux-tips/) · [uninstall](https://vupai.dev/uninstall/) · [roadmap](https://vupai.dev/roadmap/)
Contributing? See [AGENTS.md](AGENTS.md). Ideas and PRs are welcome:
[open an issue](https://github.com/itsjrsa/vupai/issues).
## License
[MIT](LICENSE). (Note: `pynput` is LGPL-3.0 and the Parakeet model weights are
CC-BY-4.0; both are runtime dependencies, not part of this repo's code.)