{"id":50773845,"url":"https://github.com/agent-rt/asrctl","last_synced_at":"2026-06-11T21:36:20.512Z","repository":{"id":355139363,"uuid":"1226575807","full_name":"agent-rt/asrctl","owner":"agent-rt","description":"macOS Apple Silicon ASR CLI (Qwen3-ASR via llama.cpp + mtmd)","archived":false,"fork":false,"pushed_at":"2026-05-02T15:32:31.000Z","size":1475,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-11T21:35:44.344Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agent-rt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-01T15:19:14.000Z","updated_at":"2026-05-02T15:32:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/agent-rt/asrctl","commit_stats":null,"previous_names":["agent-rt/asrctl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/agent-rt/asrctl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agent-rt%2Fasrctl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agent-rt%2Fasrctl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agent-rt%2Fasrctl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agent-rt%2Fasrctl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agent-rt","download_url":"https://codeload.github.com/agent-rt/asrctl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agent-rt%2Fasrctl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34219510,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-11T21:36:19.817Z","updated_at":"2026-06-11T21:36:20.506Z","avatar_url":"https://github.com/agent-rt.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# asrctl\n\nA small Zig CLI that turns audio into text on macOS Apple Silicon, with\n**two state-of-the-art ASR backends in a single 6 MB static binary**:\n\n- **Qwen3-ASR-0.6B** (Alibaba 2026, multilingual SOTA, especially Chinese)\n  via [llama.cpp](https://github.com/ggml-org/llama.cpp) + `mtmd`\n- **Whisper-large-v3-turbo Q5_0** (OpenAI multilingual SOTA)\n  via vendored [whisper.cpp](https://github.com/ggml-org/whisper.cpp)\n\nSame CLI for both backends. Real-time microphone (`asrctl listen`), neural\nsilero VAD, llama-server fallback. No Python.\n\n```sh\nasrctl audio.wav                          # → stdout, default qwen3 backend\nasrctl --backend auto audio.wav           # ★ auto-pick by language (zh→qwen3, else→whisper)\nasrctl --backend whisper audio.wav        # use whisper-large-v3-turbo\nasrctl audio.wav -o out.txt               # → file\nasrctl --server-url http://127.0.0.1:8080 audio.wav   # qwen3 via llama-server\nasrctl listen                             # mic → text stream (Ctrl-C to stop)\nasrctl listen --backend whisper --vad silero  # full SOTA stack\n```\n\nEN: 5 s wav → ~1.2 s wall time. ZH: same. See [`bench/`](bench/).\n\n## Status\n\n- ✅ v0.1: wav transcription, in-process + llama-server fallback.\n- ✅ v0.2: real-time microphone streaming via `asrctl listen`.\n- ✅ v0.3: silero neural VAD via vendored whisper.cpp (`--vad silero`).\n- ✅ v0.4: dual ASR backends — Qwen3-ASR + whisper-large-v3-turbo\n  (`--backend qwen3|whisper`).\n- ✅ v0.5: whisper streaming partial words during speech (`listen --partial`).\n- ✅ v0.6: whisper `audio_ctx` scaling — partial inference -23% wall time.\n- ✅ v0.7: whisper model picker + dual-model partial preview\n  (`--whisper-model`, auto-loads tiny for partials → ~63 ms vs ~1 s).\n- ✅ v0.8: `--backend auto` — tiny whisper probes language, routes Chinese to\n  Qwen3 / others to whisper. Zero-config, optimal-quality default.\n- ✅ v0.9: `listen --backend auto` — first-utterance probe, then sticks for\n  the session. Zero-config live mic, too.\n- See [`docs/REQ.md`](docs/REQ.md) for the full requirements \u0026 milestones.\n\n## Backend comparison\n\n| | Qwen3-ASR (default) | Whisper-large-v3-turbo |\n| --- | --- | --- |\n| Repo | `ggml-org/Qwen3-ASR-0.6B-GGUF` | `ggerganov/whisper.cpp` |\n| Disk | ~1.5 GB (model + mmproj) | ~547 MB (Q5_0) |\n| 5s wav (warm) | ~1.2 s | ~2.2 s |\n| 17s wav (warm) | ~1.7 s | ~2.2 s |\n| Strengths | Faster; Chinese; full-width punctuation | LM context recovery; less hallucination on phonetics |\n| Server fallback | ✅ via `--server-url` | ❌ (no llama-server equivalent) |\n\n## Limitations\n\n- macOS Apple Silicon only. No Linux / Intel Mac / Windows.\n- `.wav` only (`.mp3` / `.m4a` planned for v0.3+).\n- `\u003casr_text\u003e` output protocol is a Qwen3-ASR specific quirk; mtmd marks\n  audio support as \"experimental\" upstream.\n\n## Install\n\n### Pre-built (recommended)\n\n```sh\nbrew install agent-rt/tap/asrctl\n```\n\nThis is a fully self-contained binary — only depends on macOS system\nframeworks (Metal / Foundation / Accelerate). No `brew install llama.cpp`\nrequired at runtime.\n\n### Build from source\n\n```sh\nbrew install zig cmake          # cmake compiles the bundled llama.cpp\ngit clone https://github.com/agent-rt/asrctl\ncd asrctl\nzig build -Doptimize=ReleaseFast\n./zig-out/bin/asrctl --help\n```\n\n- **Zig** 0.16.0 (new `std.Io` API).\n- **CMake** to compile the vendored llama.cpp into static libraries.\n- **macOS Apple Silicon** (M1/M2/M3/M4) with Xcode Command Line Tools.\n- (No runtime dependency on `curl` — downloads use Zig's std.http.)\n\nThe first `zig build` takes ~5 minutes to compile llama.cpp + ggml. Subsequent\nbuilds reuse the cmake cache.\n\n### Build\n\n```sh\ngit clone https://github.com/agent-rt/asrctl\ncd asrctl\nzig build -Doptimize=ReleaseFast\n./zig-out/bin/asrctl --help\n```\n\n### Pre-pull the model (optional)\n\n```sh\nasrctl model pull   # downloads Qwen3-ASR-0.6B-Q8_0.gguf + mmproj (~1.5 GB)\nasrctl model path   # show resolved cache paths\n```\n\nBy default the cache lives at `$HF_HOME` / `$XDG_CACHE_HOME/huggingface` /\n`~/.cache/huggingface`, matching `huggingface_hub` conventions. If a Python\ntoolchain (or `llama-mtmd-cli -hf`) already populated that cache, asrctl\nreuses the existing files — no re-download.\n\n## Usage\n\n```\nasrctl \u003cwav-file\u003e [options]            transcribe a wav file\nasrctl model path                      print resolved model path\nasrctl model pull                      download model + mmproj from HF\nasrctl version                         print version\nasrctl help                            show this help\n\nTranscribe options:\n  -o, --output PATH    write text to file instead of stdout\n      --model PATH     override model gguf path\n      --server-url URL forward to a running llama-server instead of\n                       loading the model in-process\n      --threads N      CPU threads (default 4)\n  -v, --verbose        print timing/diagnostic info to stderr\n\nEnvironment:\n  HF_HOME              HuggingFace cache root (default ~/.cache/huggingface)\n  HF_ENDPOINT          HF mirror, e.g. https://hf-mirror.com\n```\n\nExit codes: `0` ok / `1` user error / `2` internal / `3` inference / `4` server.\n\n### Live microphone (`listen`)\n\n```sh\nasrctl listen                                     # qwen3 + energy VAD\nasrctl listen --backend auto --vad silero         # ★ zero-config: probe → pick → stick\nasrctl listen --vad silero                        # neural VAD only\nasrctl listen --backend whisper --partial         # whisper + LIVE PARTIAL words\nasrctl listen --backend whisper --partial --vad silero   # full SOTA stack\nasrctl listen -o transcript.log                   # append each utterance as a line\n```\n\n**Live partial words** (whisper backend only) — see text appear *while* the\nuser is still speaking, redrawn in dim text on the same line. When the\nutterance ends (silence cut), the line commits in normal weight + newline.\n\nBy default a tiny whisper (~31 MB) is loaded alongside the large model so\npartials render at ~63 ms each instead of ~1 s. Tune via:\n- `--whisper-model {tiny,base,small,medium,large-v3-turbo}` — main / final\n- `--whisper-partial-model NAME` — override the auto-picked partial\n- `--partial-ms 500` — cadence\n\n```\nlistening (Ctrl-C to stop, partial=on)…\nhello world how are                  ← tiny redraws every 500ms during speech\nhello world, how are you?            ← large commits on silence\n```\n\nVAD backends:\n- **`energy`** (default): RMS threshold. Fast, zero deps. Quiet rooms only.\n- **`silero`**: neural VAD via vendored whisper.cpp. Robust to fans / ambient\n  noise. Auto-downloads the 885 KB silero-v5 model on first use. CPU-only;\n  ~1 ms per 32 ms frame in ReleaseFast (negligible overhead).\n\nPipeline: 16 kHz mono PCM from CoreAudio's AudioQueue → energy-based VAD\n(RMS threshold + silence-duration cut) → for each detected utterance, the\nloaded model runs `mtmd_bitmap_init_from_audio` + `mtmd_helper_eval_chunks`\n+ greedy sampling → text printed.\n\nThe model loads once on `listen` start; per-utterance latency is just the\nASR encode/decode (~0.3-0.6 s for short phrases). First run prompts macOS\nfor microphone permission.\n\nCaveat: Qwen3-ASR is not natively streaming — partial words are not emitted\nduring speech. The listen mode segments by silence and transcribes the\nfinished utterance.\n\n### Server fallback\n\nIf you are processing many files and want to amortize the model load + Metal\nkernel JIT cost, run llama-server once and point asrctl at it:\n\n```sh\nllama-server -hf ggml-org/Qwen3-ASR-0.6B-GGUF --port 8765 \u0026\n\nfor f in *.wav; do\n  asrctl --server-url http://127.0.0.1:8765 \"$f\"\ndone\n```\n\nThe server path is 3–5× faster per request because each fresh in-process run\nre-pays the ~1 s fixed startup cost.\n\n## Architecture\n\nSingle Zig binary, ~5 modules under [`src/`](src/):\n\n- `main.zig` — entry, subcommand dispatch, stdout/stderr routing\n- `cli.zig` — argument parser\n- `hf.zig` — HuggingFace cache resolver + std.http downloader\n- `asr.zig` — in-process pipeline: load → mtmd → eval → sample → parse\n- `server.zig` — HTTP fallback via `llama-server` `/v1/audio/transcriptions`\n- `errors.zig` — wrap internal errors into human-readable diagnostics\n\n`build.zig` shells out to `cmake` to compile the vendored llama.cpp + ggml +\nmtmd into static `.a` files (one-time, cached), then links them statically\ninto asrctl. `GGML_METAL_EMBED_LIBRARY=ON` puts the Metal shader source\ninline so we don't need to ship a separate `default.metallib`.\n\nThe transcription pipeline (in `asr.zig`) follows the same shape as\n`llama-mtmd-cli`:\n\n```\nggml_backend_load_all_from_path  →  llama_backend_init\n  →  llama_model_load_from_file (n_gpu_layers=99)\n  →  llama_init_from_model\n  →  mtmd_init_from_file(mmproj, model)\n  →  mtmd_helper_bitmap_init_from_file(wav)   # miniaudio handles wav decode\n  →  prompt: \"\u003c|im_start|\u003euser\\nTranscribe.\u003c__media__\u003e\u003c|im_end|\u003e...\"\n  →  mtmd_tokenize → mtmd_helper_eval_chunks\n  →  greedy sampling loop until EOG\n  →  parse \"language X\u003casr_text\u003eY\" output\n```\n\n## Benchmark\n\nApple M2 Pro, see [`bench/results-2026-05-02.txt`](bench/results-2026-05-02.txt).\n\n| Sample | Audio | In-process | Via server (warm) |\n| --- | --- | --- | --- |\n| EN short | 5.4 s | ~1.2 s | ~0.3 s |\n| EN long | 17.0 s | ~1.7 s | ~0.6 s |\n| ZH short | 3.7 s | ~1.2 s | ~0.3 s |\n| ZH long | 17.2 s | ~1.7 s | ~0.6 s |\n\nReproduce:\n\n```sh\nbench/bench.sh                                          # in-process\nllama-server -hf ggml-org/Qwen3-ASR-0.6B-GGUF --port 8765 \u0026\nbench/bench.sh --server http://127.0.0.1:8765\n```\n\n## Project history\n\nThe `docs/` directory has the full design trail:\n\n- [`REQ.md`](docs/REQ.md) — requirements + milestones\n- [`M0-spike-report.md`](docs/M0-spike-report.md) — initial feasibility spike\n- [`M2-report.md`](docs/M2-report.md) — first end-to-end transcription\n- [`M3-report.md`](docs/M3-report.md) — CLI completion + HF cache\n- [`M4-report.md`](docs/M4-report.md) — server fallback\n- [`llama.cpp.zig-research.md`](docs/llama.cpp.zig-research.md) — reference repo notes\n- [`MLX.zig-research.md`](docs/MLX.zig-research.md) — alternative path that wasn't taken\n\n## License\n\nMIT (matching upstream llama.cpp).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagent-rt%2Fasrctl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagent-rt%2Fasrctl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagent-rt%2Fasrctl/lists"}