{"id":48740907,"url":"https://github.com/crispstrobe/crispasr","last_synced_at":"2026-07-04T16:00:51.227Z","repository":{"id":349410655,"uuid":"1195602794","full_name":"CrispStrobe/CrispASR","owner":"CrispStrobe","description":"C++ ggml runtime hub for multilingual ASR and TTS models: Cohere Transcribe, Parakeet TDT, Voxtral, Canary 1B v2, etc, plus universal forced alignment, and more","archived":false,"fork":false,"pushed_at":"2026-06-28T18:52:00.000Z","size":38269,"stargazers_count":361,"open_issues_count":36,"forks_count":55,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-06-28T19:20:25.427Z","etag":null,"topics":["cohere-transcribe","cohere-transcribe-03-2026","ggml","parakeet","speech-recognition","speech-to-text","stt","transcription","voxtral","whisper-cpp"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CrispStrobe.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.md","contributing":"docs/contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"AUDIT.md","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-03-29T21:24:48.000Z","updated_at":"2026-06-28T18:53:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/CrispStrobe/CrispASR","commit_stats":null,"previous_names":["crispstrobe/cohere-whisper.cpp","crispstrobe/crispasr"],"tags_count":56,"template":false,"template_full_name":null,"purl":"pkg:github/CrispStrobe/CrispASR","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrispStrobe%2FCrispASR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrispStrobe%2FCrispASR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrispStrobe%2FCrispASR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrispStrobe%2FCrispASR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CrispStrobe","download_url":"https://codeload.github.com/CrispStrobe/CrispASR/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CrispStrobe%2FCrispASR/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35127443,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-04T02:00:05.987Z","response_time":113,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cohere-transcribe","cohere-transcribe-03-2026","ggml","parakeet","speech-recognition","speech-to-text","stt","transcription","voxtral","whisper-cpp"],"created_at":"2026-04-12T07:08:04.723Z","updated_at":"2026-07-04T16:00:51.212Z","avatar_url":"https://github.com/CrispStrobe.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CrispASR\n\n**One C++ binary, 36 ASR backends + 23 TTS engines + multilingual text translation, zero Python dependencies.**\n\nCrispASR started as a fork of [whisper.cpp](https://github.com/ggml-org/whisper.cpp) and extends that base into a **unified speech engine** called `crispasr`, backed by full ggml C++ runtimes for major open-weights ASR *and* TTS architectures. One build, one binary, one consistent CLI — pick the backend at the command line or let CrispASR auto-detect it from your GGUF file. See [Text-to-Speech](#text-to-speech-tts) for the TTS side.\n\n```console\n$ crispasr -m ggml-base.en.bin          -f samples/jfk.wav                    # OpenAI Whisper\n$ crispasr -m parakeet-tdt-0.6b.gguf    -f samples/jfk.wav                    # NVIDIA Parakeet\n$ crispasr -m canary-1b-v2.gguf         -f samples/jfk.wav                    # NVIDIA Canary\n$ crispasr -m voxtral-mini-3b-2507.gguf -f samples/jfk.wav                    # Mistral Voxtral\n$ crispasr --backend qwen3 -m auto      -f samples/jfk.wav                    # -m auto downloads\n$ crispasr --backend kokoro -m auto --tts \"Hello world\" --tts-output out.wav  # TTS\n```\n\nNo Python. No PyTorch. No separate per-model binary. No `pip install`. Just one C++ binary and a GGUF file.\n\n**Browser**: All backends compile to WebAssembly (4.3 MB) via `build-wasm.sh`.\nMultithreaded, runs entirely client-side with COOP/COEP headers.\n\n**Demo**: [HuggingFace Space](https://huggingface.co/spaces/cstr/CrispASR) —\nlive transcription + TTS + language detection, auto-deployed from `hf-space/`.\n\n### What's new (v0.8.3)\n\n- **Audio format expansion (v0.8.3):** `.opus`, `.webm`, `.au`, `.amr`, `.m4a`/`.aac` decode — no ffmpeg. Apple platforms use AudioToolbox natively for AAC/M4A/ALAC/CAF/AIFF.\n- **qwen3-tts: GQA_NATIVE + chunked codec (v0.8.3, #183):** O(N²)→O(N) scaling on Vulkan; RTF stays ~0.5 at any text length. Chunked codec decode caps VRAM peak. Scratch scheduler reset prevents cross-request memory bloat.\n- **vibevoice TTS on Vulkan/CUDA (v0.8.3, #184):** fixed segfault on AMD RDNA4 (strided view fix) and illegal memory access on CUDA (stale gallocr state in bucket cache).\n- **chatterbox: long-text + turbo emotion tags (v0.8.2, #182):** sentence-chunked synthesis for arbitrarily long input; `[laugh]`, `[whispering]`, `[angry]` etc. drive turbo prosody.\n- **Parakeet long-audio = NeMo-exact (#89):** non-JA models default to single full-attention pass, byte-for-byte identical to upstream NeMo; clips past 300 s are silence-split with no boundary duplicates.\n- **Hotwords (#98):** `--hotwords \"Tokyo,CrispASR\"` for CTC/TDT contextual biasing + LLM prompt injection.\n- **Global diarization (#110):** `--diarize-method sherpa` / `pyannote` runs once on full audio for consistent speaker IDs.\n- **Generation controls:** `--seed`, `--beam-size`, `--frequency-penalty`, `--max-new-tokens` wired through all backends.\n\n### Ecosystem\n\n| Project | What it does |\n|---|---|\n| **[CrispASR](https://github.com/CrispStrobe/CrispASR)** | This repo — C++ speech engine. 36 ASR backends + 23 TTS engines, CLI + HTTP server + C-ABI + Python/Rust/Dart/Go/Ruby bindings. |\n| **[CrisperWeaver](https://github.com/CrispStrobe/CrisperWeaver)** | Cross-platform Flutter transcription app built on CrispASR. Desktop + mobile, all 10 backends, model browser with download queue, mic capture, SRT/VTT/JSON export, diarization, batch processing. Fully offline. |\n| **[CrispEmbed](https://github.com/CrispStrobe/CrispEmbed)** | Text embedding engine via ggml — same philosophy as CrispASR but for retrieval. 10 architectures (XLM-R, Qwen3-Embed, Gemma3, ModernBERT, ...), dense + sparse + ColBERT + reranking. 9.5x faster than ONNX on CPU, GPU via CUDA/Metal/Vulkan. Python/Rust/Dart bindings. |\n| **[Susurrus](https://github.com/CrispStrobe/Susurrus)** | Python ASR GUI with 9 backends (faster-whisper, mlx-whisper, voxtral, insanely-fast-whisper, ...). The Python counterpart to CrispASR's C++ approach. |\n\n---\n\n## Table of contents\n\n- [Supported backends](#supported-backends) — [ASR](#asr-backends) + [TTS](#text-to-speech-models) + [translation](#translation) + [post-processing](#post-processing-models)\n- [Feature matrix](#feature-matrix)\n- [Install \u0026 build](#install--build) — quick install (full guide in [docs/install.md](docs/install.md))\n- [Quick start — ASR](#quick-start)\n- [**Text-to-Speech (TTS)**](docs/tts.md) — Kokoro, Qwen3-TTS, VibeVoice, Orpheus, Chatterbox, IndexTTS, VoxCPM2, CosyVoice3, CSM, Dia, Zonos, Bark, Piper, MeloTTS, and more\n- [Streaming \u0026 live transcription](docs/streaming.md)\n- [Server mode (HTTP API)](docs/server.md)\n- [CLI reference](docs/cli.md) — flags, VAD, CTC alignment, output formats, auto-download, audio formats\n- [Language bindings](docs/bindings.md) — Python / Rust / Dart / Go / Java / JavaScript / Ruby / mobile\n- [Architecture](docs/architecture.md) — layered layout, `src/core/` primitives, regression discipline\n- [Contributing — adding a new backend](docs/contributing.md) — 5-file recipe, ground-truth diff workflow\n- [Regression matrix](docs/regression-matrix.md) — `tools/test-all-backends.py` capability tiers\n- [Quantize models](docs/quantize.md) — `crispasr-quantize` for all backends\n- [GPU backend selection](#gpu-backend-selection)\n- [Debugging \u0026 profiling](#debugging--profiling)\n- [Credits](#credits)\n\n---\n\n## Supported backends\n\nCrispASR ships **36 ASR backends** for transcription/translation and\n**23 TTS engines** for synthesis. Pick at the CLI with `--backend NAME`,\nor omit it to let the binary auto-detect from the GGUF metadata. Jump\nto the [TTS table](#text-to-speech-models) for the synthesis side.\n\n### ASR backends\n\n| Backend | Model | Architecture | Languages | License |\n|---|---|---|---|---|\n| **whisper** | [`ggml-base.en.bin`](https://huggingface.co/ggerganov/whisper.cpp/) and all OpenAI Whisper variants | Encoder-decoder transformer | 99 | MIT |\n| **whisper** | [`distil-whisper/distil-large-v3`](https://huggingface.co/cstr/distil-large-v3-GGUF) | Distilled Whisper: 32L encoder + 2L decoder (6.3x faster) | English | MIT |\n| **parakeet** | [`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) | FastConformer + TDT | 25 EU (auto-detect) | CC-BY-4.0 |\n| **parakeet** | [`nvidia/parakeet-tdt-0.6b-v2`](https://huggingface.co/cstr/parakeet-tdt-0.6b-v2-GGUF) | FastConformer + TDT, original Open ASR Leaderboard topper | en (mixed-case + punct) | CC-BY-4.0 |\n| **parakeet** | [`nvidia/parakeet-tdt-1.1b`](https://huggingface.co/cstr/parakeet-tdt-1.1b-GGUF) | 42L FastConformer + TDT, larger English variant | en (lowercase) | CC-BY-4.0 |\n| **parakeet** | [`nvidia/parakeet-tdt_ctc-110m`](https://huggingface.co/cstr/parakeet-tdt_ctc-110m-GGUF) | 17L FastConformer + TDT+CTC hybrid; smallest variant, auto-CTC decode | en | CC-BY-4.0 |\n| **parakeet** | [`nvidia/parakeet-tdt_ctc-1.1b`](https://huggingface.co/cstr/parakeet-tdt_ctc-1.1b-GGUF) | 42L FastConformer + TDT+CTC hybrid; largest, mixed-case + punct | en | CC-BY-4.0 |\n| **parakeet** | [`nvidia/parakeet-tdt_ctc-0.6b-ja`](https://huggingface.co/cstr/parakeet-tdt-0.6b-ja-GGUF) | FastConformer-TDT-CTC, xscaling, 80 mels | Japanese | CC-BY-4.0 |\n| **reazonspeech** | [`reazon-research/reazonspeech-nemo-v2`](https://huggingface.co/cstr/reazonspeech-nemo-v2-GGUF) | FastConformer-RNNT, local attn (w=256), 80 mels, 619M params | Japanese | Apache-2.0 |\n| **fastconformer-ctc** | [`nvidia/parakeet-ctc-0.6b`](https://huggingface.co/cstr/parakeet-ctc-0.6b-GGUF) | 24L FastConformer + CTC, 80 mels (same arch as fc-ctc-xlarge) | en | CC-BY-4.0 |\n| **fastconformer-ctc** | [`nvidia/parakeet-ctc-1.1b`](https://huggingface.co/cstr/parakeet-ctc-1.1b-GGUF) | 42L FastConformer + CTC, 80 mels | en | CC-BY-4.0 |\n| **fastconformer-ctc** | [`grider-transwithai/parakeet-ctc-1.1b-ja`](https://huggingface.co/cstr/parakeet-ctc-1.1b-ja-GGUF) | 42L FastConformer + CTC, 80 mels, Japanese fine-tune | Japanese | Apache-2.0 |\n| **canary** | [`nvidia/canary-1b-v2`](https://huggingface.co/nvidia/canary-1b-v2) | FastConformer + Transformer decoder | 25 EU (explicit `-sl/-tl`) | CC-BY-4.0 |\n| **lfm2-audio** | [`LiquidAI/LFM2.5-Audio-1.5B`](https://huggingface.co/cstr/lfm2-audio-1.5b-GGUF) | FastConformer + LFM2 hybrid conv+attention backbone (ASR+TTS) | en | LFM Open v1.0 |\n| **lfm2-audio** | [`LiquidAI/LFM2.5-Audio-1.5B-JP`](https://huggingface.co/cstr/lfm2-audio-1.5b-jp-GGUF) | FastConformer + LFM2 hybrid conv+attention backbone (ASR+TTS) | ja | LFM Open v1.0 |\n| **mini-omni2** | [`gpt-omni/mini-omni2`](https://huggingface.co/gpt-omni/mini-omni2) | Whisper-small + Qwen2-0.5B (ASR+TTS+S2S) | en | MIT |\n| **cohere** | [`CohereLabs/cohere-transcribe-03-2026`](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026) | Conformer + Transformer | 13 | Apache-2.0 |\n| **cohere** | [`efwkjn/cohere-asr-ja-v0.1`](https://huggingface.co/TransWithAI/cohere-transcribe-ja-v0.1-GGUF) | Japanese fine-tune of cohere-transcribe-03-2026 (TedX/JSUT-tuned) | Japanese | Apache-2.0 |\n| **granite** | [`ibm-granite/granite-speech-{3.2-8b,3.3-2b,3.3-8b}`](https://huggingface.co/ibm-granite/granite-speech-3.3-2b), [`granite-4.0-1b-speech`](https://huggingface.co/ibm-granite/granite-4.0-1b-speech) | Conformer + Q-Former + Granite LLM (μP) ([more](docs/architecture.md#granite--granite-41--granite-41-plus--granite-41-nar)) | en fr de es pt ja | Apache-2.0 |\n| **granite-4.1** | [`ibm-granite/granite-speech-4.1-2b`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b) | 16L Conformer + Q-Former + Granite LLM; single ggml graph ([more](docs/architecture.md#granite--granite-41--granite-41-plus--granite-41-nar)) | en fr de es pt ja | Apache-2.0 |\n| **granite-4.1-plus** | [`ibm-granite/granite-speech-4.1-2b-plus`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus) | 4.1 + hidden-state concat; punctuated output ([more](docs/architecture.md#granite--granite-41--granite-41-plus--granite-41-nar)) | en fr de es pt | Apache-2.0 |\n| **granite-4.1-nar** | [`ibm-granite/granite-speech-4.1-2b-nar`](https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar) | Non-autoregressive: single LLM forward + slot argmax ([more](docs/architecture.md#granite--granite-41--granite-41-plus--granite-41-nar)) | en fr de es pt | Apache-2.0 |\n| **fastconformer-ctc** | [`nvidia/stt_en_fastconformer_ctc_large`](https://huggingface.co/nvidia/stt_en_fastconformer_ctc_large) | FastConformer + CTC (NeMo family, all sizes) | en | CC-BY-4.0 |\n| **voxtral** | [`mistralai/Voxtral-Mini-3B-2507`](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) | Whisper encoder + Mistral 3B LLM | 8 | Apache-2.0 |\n| **voxtral4b** | [`mistralai/Voxtral-Mini-4B-Realtime-2602`](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602) | Causal encoder + 3.4B LLM, sliding window | 13, realtime streaming | Apache-2.0 |\n| **qwen3** | [`Qwen/Qwen3-ASR-0.6B`](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) | Whisper-style audio encoder + Qwen3 0.6B LLM | 30 + 22 Chinese dialects | Apache-2.0 |\n| **qwen3-1.7b** | [`Qwen/Qwen3-ASR-1.7B`](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) | Whisper-style audio encoder + Qwen3 1.7B LLM | 30 + 22 Chinese dialects | Apache-2.0 |\n| **qwen3-ja-anime** | [`jaykwok/Qwen3-ASR-1.7B-JA-Anime-Galgame-hf`](https://huggingface.co/jaykwok/Qwen3-ASR-1.7B-JA-Anime-Galgame-hf) | Qwen3-ASR-1.7B fine-tuned for Japanese anime/galgame speech | ja + 30 langs | Apache-2.0 |\n| **mega-asr** | [`zhifeixie/Mega-ASR`](https://huggingface.co/zhifeixie/Mega-ASR) | Qwen3-ASR-1.7B + merged robustness LoRA; always-on robust path | noisy / degraded speech | Apache-2.0 |\n| **higgs-stt** | [`bosonai/higgs-audio-v3-stt`](https://huggingface.co/bosonai/higgs-audio-v3-stt) | Whisper-large-v3 encoder (4 s chunked) + Qwen3-1.7B LLM ([more](docs/architecture.md#higgs-stt)) | en | Apache-2.0 |\n| **wav2vec2** | [`jonatasgrosman/wav2vec2-large-xlsr-53-english`](https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-english) | CNN + 24L transformer + CTC head (any Wav2Vec2ForCTC) | per-model | Apache-2.0 |\n| **wav2vec2** | [`facebook/data2vec-audio-base-960h`](https://huggingface.co/cstr/data2vec-audio-960h-GGUF) | Data2Vec Audio (79 MB Q4_K) | English | Apache-2.0 |\n| **wav2vec2** | [`facebook/hubert-large-ls960-ft`](https://huggingface.co/cstr/hubert-large-ls960-ft-GGUF) | HuBERT Large (212 MB Q4_K) | English | Apache-2.0 |\n| **glm-asr** | [`zai-org/GLM-ASR-Nano-2512`](https://huggingface.co/zai-org/GLM-ASR-Nano-2512) | Whisper encoder + 4-frame projector + Llama 1.5B (GQA) | Mandarin (+ Chinese dialects), English, Cantonese | MIT |\n| **kyutai-stt** | [`kyutai/stt-1b-en_fr`](https://huggingface.co/kyutai/stt-1b-en_fr) | Mimi codec (SEANet + RVQ) + 16L causal LM | en, fr | MIT |\n| **firered-asr** | [`FireRedTeam/FireRedASR2-AED`](https://huggingface.co/FireRedTeam/FireRedASR2-AED) | Conformer + CTC + beam search; also LID (120 langs) | Mandarin, English, 20+ Chinese dialects | Apache-2.0 |\n| **moonshine** | [`UsefulSensors/moonshine-{tiny,base}`](https://huggingface.co/cstr/moonshine-base-GGUF) | Conv + 6L enc + 6L dec; multilingual variants | English + 6 langs | MIT |\n| **moonshine\u0026#8209;de** | [`fidoriel/moonshine-base-de`](https://huggingface.co/cstr/moonshine-base-de-fidoriel-GGUF) | German fine-tune of moonshine-base (6.9% WER CV22) | German | CC\u0026#8209;BY\u0026#8209;NC\u0026#8209;SA\u0026#8209;4.0 |\n| **moonshine\u0026#8209;tiny\u0026#8209;de** | [`fidoriel/moonshine-tiny-de`](https://huggingface.co/cstr/moonshine-tiny-de-fidoriel-GGUF) | German fine-tune of moonshine-tiny (11.4% WER CV22) | German | CC\u0026#8209;BY\u0026#8209;NC\u0026#8209;SA\u0026#8209;4.0 |\n| **moonshine-streaming** | [`UsefulSensors/moonshine-streaming-{tiny,small,medium}`](https://huggingface.co/cstr/moonshine-streaming-tiny-GGUF) | Streaming: sliding-window encoder + AR decoder (34–245M) | English | MIT |\n| **gemma4-e2b** | [`google/gemma-4-E2B-it`](https://huggingface.co/cstr/gemma4-e2b-it-GGUF) | USM Conformer 12L + Gemma4 LLM 35L (GQA, PLE) | 140+ langs | Apache-2.0 |\n| **gemma4-e4b** | [`google/gemma-4-E4B-it`](https://huggingface.co/cstr/gemma4-e4b-it-GGUF) | Same USM Conformer 12L + larger Gemma4 LLM 42L (GQA, PLE); runs on `--backend gemma4-e2b` | 140+ langs | Apache-2.0 |\n| **omniasr** | [`omniASR-CTC-1B-v2`](https://huggingface.co/cstr/omniASR-CTC-1B-v2-GGUF) | wav2vec2 CNN + 48L transformer + CTC ([more](docs/architecture.md#omniasr-ctc--llm--unlimited)) | **1600+** | Apache-2.0 |\n| **omniasr\u0026#8209;300m** | [`omniASR-CTC-300M-v2`](https://huggingface.co/cstr/omniASR-CTC-300M-v2-GGUF) | Same arch, 24L, ~194 MB Q4_K; auto-chunks \u003e7 s ([more](docs/architecture.md#omniasr-ctc--llm--unlimited)) | **1600+** | Apache-2.0 |\n| **omniasr-llm** | [`omniASR-LLM-300M-v2`](https://huggingface.co/cstr/omniasr-llm-300m-v2-GGUF) | Same encoder + 12L LLaMA decoder ([more](docs/architecture.md#omniasr-ctc--llm--unlimited)) | **1600+** | Apache-2.0 |\n| **omniasr-llm** | [`omniASR-LLM-Unlimited-300M-v2`](https://huggingface.co/cstr/omniasr-llm-unlimited-300m-v2-GGUF) | Streaming: 15s segment protocol, unlimited audio ([more](docs/architecture.md#omniasr-ctc--llm--unlimited)) | **1600+** | Apache-2.0 |\n| **vibevoice** | [`microsoft/VibeVoice-ASR`](https://huggingface.co/cstr/vibevoice-asr-GGUF) | σ-VAE ConvNeXt + Qwen2.5-7B ([more](docs/architecture.md#vibevoice)) | 50+ | MIT |\n| **mimo-asr** | [`XiaomiMiMo/MiMo-V2.5-ASR`](https://huggingface.co/cstr/mimo-asr-GGUF) | 6L transformer + 36L Qwen2 LM + RVQ codec ([more](docs/architecture.md#mimo-asr)) | Mandarin + dialects + English | MIT |\n| **ark-asr** ⚠️*experimental/WIP* | [`cstr/ark-asr-3b-GGUF`](https://huggingface.co/cstr/ark-asr-3b-GGUF) (base [`AutoArk-AI/ARK-ASR-3B`](https://huggingface.co/AutoArk-AI/ARK-ASR-3B)) | Whisper-large-v3 enc (partial RoPE) + Qwen2.5-3B LM ([more](docs/architecture.md#ark-asr)) | 19 (zh, en, de, ja, fr, ko, es, pl, it, ro, hu, cs, nl, fi, hr, sk, sl, et, lt) | see base |\n| **moss-audio** | [`OpenMOSS-Team/MOSS-Audio-4B-Instruct`](https://huggingface.co/cstr/MOSS-Audio-4B-Instruct-GGUF) | 32L Whisper encoder + DeepStack 3-tap + 36L Qwen3 LM; audio understanding + ASR ([more](docs/architecture.md#moss-audio)) | zh, en | Apache-2.0 |\n| **moss-transcribe** | [`OpenMOSS-Team/MOSS-Transcribe-preview-2B`](https://huggingface.co/cstr/MOSS-Transcribe-preview-2B-GGUF) | Qwen3-Omni audio encoder (32L, windowed attn) + GatedMLP adapter + Qwen3-1.7B LM; ASR ([more](docs/architecture.md#moss-transcribe)) | zh, en | Apache-2.0 |\n| **funasr** | [`FunAudioLLM/Fun-ASR-Nano-2512`](https://huggingface.co/cstr/funasr-nano-GGUF) | 70-block SANM encoder + 2-block Transformer adaptor + Qwen3-0.6B LLM | zh, yue, en, ja, ko | FunASR Model License v1.1 (commercial OK w/ attribution) |\n| **fun-asr-mlt-nano** | [`FunAudioLLM/Fun-ASR-MLT-Nano-2512`](https://huggingface.co/cstr/funasr-mlt-nano-GGUF) | Same architecture, multilingual decoder | 31 langs incl. de, fr, es, pt, ru, ar, hi, vi, th, ko | FunASR Model License v1.1 |\n| **paraformer** | [`funasr/paraformer-zh`](https://huggingface.co/cstr/paraformer-zh-GGUF) | 50-block SANM encoder + CIF predictor + 16-block NAR decoder (single-pass, non-autoregressive); character-level vocab (8404); 220M params | zh, en | FunASR Model License (commercial OK w/ attribution) |\n| **sensevoice** | [`FunAudioLLM/SenseVoiceSmall`](https://huggingface.co/cstr/sensevoice-small-GGUF) | 70-block SANM encoder + CTC head; emits transcript + language ID + emotion + audio-event in one forward pass (non-AR, 15× faster than Whisper-Large); structured C ABI + `-oj` JSON expose the four tags as separate fields | 50+ langs; native LID + emotion + audio-event tags | FunASR Model License v1.1 |\n\n### Text-to-Speech models\n\nSynthesis backends, driven by the `--tts` flag and a `--tts-output PATH.wav`.\nSee the dedicated [Text-to-Speech](#text-to-speech-tts) section below for\nquick-start commands and engine selection guidance.\n\n| Backend | Models | Architecture | Languages | License |\n|---------|--------|-------------|-----------|---------|\n| **vibevoice-tts** | [`VibeVoice-Realtime-0.5B`](https://huggingface.co/cstr/vibevoice-realtime-0.5b-GGUF), [`VibeVoice-1.5B`](https://huggingface.co/cstr/vibevoice-1.5b-GGUF) | DPM-Solver++ + σ-VAE decoder; voice presets or cloning | en, zh | MIT |\n| **kugelaudio** | [`kugelaudio-0-open`](https://huggingface.co/cstr/kugelaudio-0-open-GGUF) | Qwen2.5-7B LM + 4L DiT diffusion + acoustic VAE decoder; voice cloning | multilingual | Apache-2.0 |\n| **qwen3-tts** | [`Qwen3-TTS-12Hz-0.6B-Base`](https://huggingface.co/cstr/qwen3-tts-0.6b-base-GGUF), [`1.7B-Base`](https://huggingface.co/cstr/qwen3-tts-1.7b-base-GGUF), [`1.7B-VoiceDesign`](https://huggingface.co/cstr/qwen3-tts-1.7b-voicedesign-GGUF) | Qwen3 talker LM + 12 Hz RVQ ([more](docs/architecture.md#qwen3-tts)) | multilingual | Apache-2.0 |\n| **qwen3-tts-customvoice** | [`1.7B-CustomVoice`](https://huggingface.co/cstr/qwen3-tts-1.7b-customvoice-GGUF) | Same talker + 9 premium built-in speakers (`--voice \u003cname\u003e`); optional style via `--instruct` (e.g. \"spoke very slowly\") ([more](docs/architecture.md#qwen3-tts)) | multilingual | Apache-2.0 |\n| **melotts** | [`myshell-ai/MeloTTS`](https://github.com/myshell-ai/MeloTTS) EN_V2 | VITS2 (6L transformer + SDP/DP + transformer coupling flow + HiFi-GAN); 44.1 kHz, 102 MB + 52 MB BERT Q4_K companion (154 MB total); neural G2P; 4 EN speakers ([more](docs/architecture.md#melotts)) | en | MIT |\n| **piper** | [`rhasspy/piper`](https://github.com/rhasspy/piper) community voices | VITS (6L transformer + SDP + 4-block coupling flow + HiFi-GAN); 22 kHz mono, 30 MB F16 per voice; built-in G2P for EN/DE/FR/ES (`--g2p-dict`) | 30+ langs (built-in + espeak dlopen) | MIT |\n| **kokoro** | [`hexgrad/Kokoro-82M`](https://huggingface.co/hexgrad/Kokoro-82M) + German backbones | StyleTTS2 / iSTFTNet (82M); per-voice GGUF ([more](docs/architecture.md#kokoro)) | en, es, fr, hi, it, ja, pt, zh, de | Apache-2.0 |\n| **orpheus** | [`Orpheus-3B-FT`](https://huggingface.co/cstr/orpheus-3b-0.1-ft-GGUF) + [`SNAC 24 kHz`](https://huggingface.co/cstr/snac-24khz-GGUF) | Llama-3.2-3B + SNAC RVQ codec; 8 speakers ([more](docs/architecture.md#orpheus)) | en, de | Llama / MIT |\n| **chatterbox** | [`cstr/chatterbox-GGUF`](https://huggingface.co/cstr/chatterbox-GGUF) + turbo/kartoffelbox/lahgtna variants | T3 AR + S3Gen flow-matching ([more](docs/architecture.md#chatterbox--chatterbox-turbo--kartoffelbox-turbo--lahgtna-chatterbox)) | 23 multilingual; separate Arabic (`lahgtna-chatterbox`) and German turbo (`kartoffelbox-turbo`) fine-tunes | MIT |\n| **indextts** | [`cstr/indextts-1.5-GGUF`](https://huggingface.co/cstr/indextts-1.5-GGUF) | GPT-2 AR (24L/1280d) + Conformer conditioning + BigVGAN vocoder; voice cloning via reference audio | zh, en | Apache-2.0 |\n| **voxcpm2-tts** | [`cstr/voxcpm2-GGUF`](https://huggingface.co/cstr/voxcpm2-GGUF) | Tokenizer-free CFM diffusion AR (TSLM + RALM + LocDiT) at 48 kHz native; zero-shot + voice cloning via `--voice \u003cwav\u003e` | 30 languages | Apache-2.0 |\n| **cosyvoice3-tts** | [`cstr/cosyvoice3-0.5b-2512-GGUF`](https://huggingface.co/cstr/cosyvoice3-0.5b-2512-GGUF) | Qwen2-0.5B AR speech-token LM + DiT-CFM (10-step Euler) + HiFT (NSF + iSTFT) at 24 kHz; baked-voice zero-shot cloning via `--voice \u003cname\u003e` | 9 langs + 18 zh dialects | Apache-2.0 |\n| **csm** | [`cstr/csm-1b-GGUF`](https://huggingface.co/cstr/csm-1b-GGUF) | Sesame CSM-1B conversational TTS: Llama-3.2 1B backbone + 100M depth decoder (32-codebook RVQ) + Kyutai Mimi codec at 24 kHz ([more](docs/architecture.md#csm)) | en | Apache-2.0 |\n| **lfm2-audio** | [`cstr/lfm2-audio-1.5b-GGUF`](https://huggingface.co/cstr/lfm2-audio-1.5b-GGUF) + [`jp`](https://huggingface.co/cstr/lfm2-audio-1.5b-jp-GGUF) | LFM2.5-Audio ASR+TTS+S2S: FastConformer enc + LFM2 hybrid backbone + depthformer (8-codebook Mimi) + ISTFT detokenizer at 24 kHz; interleaved text+audio generation | en, ja | LFM Open v1.0 |\n| **dia** | [`nari-labs/Dia-1.6B`](https://huggingface.co/cstr/dia-1.6b-GGUF) | Byte-level text encoder (12L) + AR audio decoder (18L GQA + CFG) → 9 delayed DAC codebooks + 44.1 kHz DAC codec; dialogue style with `[S1]`/`[S2]` tags (use \u003e100-char prompts) | en | Apache-2.0 |\n| **zonos-tts** | [`cstr/zonos-v0.1-transformer-GGUF`](https://huggingface.co/cstr/zonos-v0.1-transformer-GGUF) + [`cstr/dac-44khz-GGUF`](https://huggingface.co/cstr/dac-44khz-GGUF) | Zyphra Zonos-v0.1: 26L GQA AR transformer (2B) + 9-codebook DAC @ 44.1 kHz; CFG-guided; voice cloning via reference WAV ([more](docs/architecture.md#zonos-tts)) | en | Apache-2.0 |\n| **bark** | [`cstr/bark-small-GGUF`](https://huggingface.co/cstr/bark-small-GGUF) | Suno Bark 3-stage GPT-2 TTS: text→semantic (12L) → coarse EnCodec (12L, 2 codebooks) → fine (12L, 8 codebooks) → EnCodec 24 kHz decoder; speaker conditioning via `.npz` prompts (`--voice \u003cfile.npz\u003e`) | multilingual | MIT |\n| **speecht5** | [`cstr/speecht5-tts-GGUF`](https://huggingface.co/cstr/speecht5-tts-GGUF) | SpeechT5 80M: char-level encoder (12L) + AR mel decoder (6L) + 5-layer conv postnet + HiFi-GAN at 16 kHz; speaker via 512-d x-vector (`--voice \u003cxvector.bin\u003e`) | en | MIT |\n| **fastpitch** | [`cstr/fastpitch-en-GGUF`](https://huggingface.co/cstr/fastpitch-en-GGUF) | NVIDIA FastPitch 60M: non-autoregressive parallel TTS — 6L encoder + duration/pitch predictors + 6L decoder + HiFi-GAN at 22 kHz; deterministic, single forward pass ([more](docs/architecture.md#fastpitch)) | en | CC-BY-4.0 |\n| **bananamind-tts** | `Banaxi-Tech/BananaMind-TTS-V2.1-Preview` | BananaMind-TTS 13M: Tacotron-lite char-level encoder (Conv+BN+BiLSTM) + AR GRU decoder with location-sensitive attention + postnet + HiFi-GAN at 22 kHz; fixed voice per locale ([more](docs/architecture.md#bananamind-tts)) | en, de | Apache-2.0 |\n| **parler-tts** | [`cstr/parler-tts-mini-v1.1-GGUF`](https://huggingface.co/cstr/parler-tts-mini-v1.1-GGUF) | Parler TTS Mini v1.1 (~900M): T5 encoder + MusicGen decoder + DAC 44.1 kHz; prompt-conditioned (describe voice in text via `--instruct`) | en | Apache-2.0 |\n| **outetts** | [`cstr/outetts-0.3-1b-GGUF`](https://huggingface.co/cstr/outetts-0.3-1b-GGUF) | OLMo-1B talker + WavTokenizer single-codebook VQ-GAN at 24 kHz; voice cloning via speaker profile JSON (`--voice \u003cspeaker.json\u003e`) | en | CC-BY-4.0 |\n| **pocket-tts** | [`cstr/pocket-tts-GGUF`](https://huggingface.co/cstr/pocket-tts-GGUF) | Kyutai Pocket TTS 100M: continuous-latent AR at 12.5 Hz + one-step LSD flow + Mimi VAE 24 kHz; voice cloning via ref audio ([more](docs/architecture.md#pocket-tts)) | en | MIT / CC-BY-4.0 |\n| **tada** | [`cstr/tada-tts-1b-GGUF`](https://huggingface.co/cstr/tada-tts-1b-GGUF) + `HumeAI/tada-3b-ml` | Llama-3.2 1B/3B backbone + per-token FM diffusion head + TADA codec at 24 kHz; 1:1 text-to-acoustic alignment; default prompt via `tada-ref.gguf`, custom voices via `--voice \u003ctada-ref.gguf\u003e` built with `models/convert-tada-ref-to-gguf.py` ([more](docs/architecture.md#tada)) | en | — |\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003eTTS feature matrix\u003c/b\u003e\u003c/summary\u003e\n\n| Backend | Voice cloning | Sampling | kHz | Auto-download | Flash attn |\n|---------|:---:|:---:|:---:|:---:|:---:|\n| vibevoice-tts | yes | temp | 24 | yes | yes |\n| qwen3-tts | yes* | temp | 24 | yes | yes |\n| kokoro | — | — | 24 | yes | — |\n| orpheus | — | temp | 24 | yes | yes |\n| chatterbox | yes | temp | 24 | yes | yes |\n| outetts | yes (JSON) | temp | 24 | yes | yes |\n| indextts | yes | temp | 24 | yes | yes |\n| voxcpm2-tts | yes | — | 48 | yes | — |\n| cosyvoice3-tts | yes | temp | 24 | yes | yes |\n| f5-tts | yes | — | 24 | yes | — |\n| csm | — | temp | 24 | yes | — |\n| dia | — | temp | 44 | yes | — |\n| bark | yes (.npz) | temp | 24 | yes | — |\n| speecht5 | yes (xvec) | — | 16 | yes | — |\n| parler-tts | — | temp | 44 | yes | — |\n| fastpitch | — | — | 22 | — | — |\n| piper | — | — | 22 | — | — |\n| pocket-tts | yes | temp | 24 | yes | — |\n| tada | yes | temp | 24 | yes | — |\n| dots-tts | yes | — | 48 | yes | — |\n\n\\* CustomVoice variant only; Base uses baked speakers via `--voice \u003cname\u003e`.\n\n\u003c/details\u003e\n\n### Translation\n\nText-to-text translation, distinct from the audio-side `--translate`\nflag (which routes audio → English text on whisper / canary / etc.).\nDriven by `--text \"...\" -sl \u003csrc\u003e -tl \u003ctgt\u003e`.\n\n| Backend | Models | Architecture | Languages | License |\n|---|---|---|---|---|\n| **m2m100** | [`facebook/m2m100_418M`](https://huggingface.co/cstr/m2m100-418m-GGUF) | 12L enc + 12L dec transformer, SentencePiece 128K ([more](docs/architecture.md#m2m100--wmt21)) | 100 langs, any-to-any | MIT |\n| **m2m100-wmt21** | [`facebook/wmt21-dense-24-wide-en-x`](https://huggingface.co/cstr/wmt21-dense-24-wide-en-x-GGUF) + [`facebook/wmt21-dense-24-wide-x-en`](https://huggingface.co/cstr/wmt21-dense-24-wide-x-en-GGUF) | Same as m2m100, scaled to 4.7B (24L enc) ([more](docs/architecture.md#m2m100--wmt21)) | English ↔ 7 langs (separate `en-x` / `x-en` checkpoints) | MIT |\n| **madlad** | [`google/madlad400-3b-mt`](https://huggingface.co/cstr/madlad400-3b-mt-GGUF) | T5 enc-dec (12L+12L, d=2048, gated-GELU, RMSNorm) ([more](docs/architecture.md#madlad)) | 419 languages | Apache-2.0 |\n\n```bash\n# m2m100 base (production-ready)\n./build/bin/crispasr --backend m2m100 -m auto \\\n    --text \"Hello world, how are you today?\" \\\n    -sl en -tl de\n# → Hallo Welt, wie bist du heute?\n\n# WMT21 dense (English ↔ X, 4.7B — auto-downloads ~2.5 GB).\n# Two separate checkpoints: en-x for English-source, x-en for\n# English-target. Pick the one matching your `-sl`/`-tl` direction\n# (or pass an explicit `-m \u003cpath\u003e` to load the other manually).\n./build/bin/crispasr --backend m2m100-wmt21 -m auto \\\n    --text \"The president said he would not attend.\" \\\n    -sl en -tl de   # uses wmt21-dense-24-wide-en-x\n\n./build/bin/crispasr --backend m2m100-wmt21 \\\n    -m models/wmt21-dense-24-wide-x-en-q4_k.gguf \\\n    --text \"Le président a dit qu'il ne serait pas présent.\" \\\n    -sl fr -tl en   # uses wmt21-dense-24-wide-x-en\n\n# MADLAD-400 3B (419 languages, bit-token-identical to Python SP)\n./build/bin/crispasr --backend madlad -m auto \\\n    --text \"Hello world.\" \\\n    -sl en -tl ta\n```\n\nFor 2-stage pipelines (e.g., ASR → m2m100), use the dedicated\n`--tr-sl` / `--tr-tl` flags; they fall back to `-sl` / `-tl` when\nunset, so single-stage standalone usage is just `-sl/-tl`.\n\n### Post-processing models\n\nWork with all backends.\n\n| Model | Task | Architecture | Languages | License | HuggingFace |\n|---|---|---|---|---|---|\n| **FireRedPunc** | Punctuation restoration | BERT-base (12L, d=768), 5 classes | Chinese + English | Apache-2.0 | [`cstr/fireredpunc-GGUF`](https://huggingface.co/cstr/fireredpunc-GGUF) |\n| **fullstop-punc** | Punctuation restoration | XLM-RoBERTa-large (24L, d=1024), 6 classes | EN, DE, FR, IT | MIT | [`cstr/fullstop-punc-multilang-GGUF`](https://huggingface.co/cstr/fullstop-punc-multilang-GGUF) |\n| **punctuate-all** | Punctuation restoration | XLM-RoBERTa-base (12L, d=768), 6 classes | 12 languages | MIT | [`cstr/punctuate-all-GGUF`](https://huggingface.co/cstr/punctuate-all-GGUF) |\n| **PCS** | Punc + truecase + SBD | XLM-RoBERTa-base (12L), 4 heads | 47 languages | Apache-2.0 | `--punc-model pcs` |\n| **truecaser\u0026#8209;lstm** | German truecasing (best) | BiLSTM char-level (2×150, 3.2 MB, 97.9% F1) | German | Apache-2.0 | `--truecase-model lstm` |\n| **truecaser\u0026#8209;crf** | German truecasing | CRF + context features (8.5 MB) | German | MIT | `--truecase-model crf` |\n| **truecaser\u0026#8209;de** | German truecasing (simple) | Statistical word-frequency (71K entries, 1.7 MB) | German | MIT | `--truecase-model auto` |\n| **CLD3** | Text language ID | Embedding-bag → FC + ReLU → softmax (~1.5 MB F32) | 109 ISO 639-1 | Apache-2.0 | [`cstr/cld3-GGUF`](https://huggingface.co/cstr/cld3-GGUF) |\n| **GlotLID-V3** | Text language ID | fastText supervised, flat softmax | 2102 ISO 639-3 + script | Apache-2.0 | [`cstr/glotlid-GGUF`](https://huggingface.co/cstr/glotlid-GGUF) |\n| **LID-176** | Text language ID | fastText supervised, hierarchical softmax | 176 ISO 639-1 | CC-BY-SA-3.0 | [`cstr/fasttext-lid176-GGUF`](https://huggingface.co/cstr/fasttext-lid176-GGUF) |\n\nAll runtimes share ggml-based inference. The speech-LLM backends (**qwen3**, **voxtral**, **voxtral4b**, **granite**, **glm-asr**, **kyutai-stt**) inject audio encoder frames directly into an autoregressive language model's input embeddings, instead of using a dedicated CTC/transducer/seq2seq decoder. The **fastconformer-ctc** backend hosts the NeMo FastConformer-CTC standalone ASR family — `stt_en_fastconformer_ctc_{large,xlarge,xxlarge}` and the architecturally-identical `parakeet-ctc-{0.6b,1.1b}` (different training data + tokenizer, same encoder + head shape) — with greedy CTC decoding. Same C++ runtime as the canary-ctc aligner.\n\n## Feature matrix\n\nRun `crispasr --list-backends` to see it live. Each backend declares capabilities at runtime; if you ask for a feature the selected backend does not support, CrispASR prints a warning and silently ignores the flag.\n\n**Sortable / filterable view:** [`docs/feature-matrix.html`](docs/feature-matrix.html) — click any column header to sort, type to filter rows, click cap pills to require a capability. Generated from `crispasr --list-backends-json` (single source of truth — drift impossible). Regenerate via `python tools/gen-feature-matrix.py`. A Markdown twin lives at [`docs/feature-matrix.md`](docs/feature-matrix.md).\n\nThe static table below is a curated subset focusing on the ASR backends and the cross-cutting features that matter for ASR pipelines. The full 39-backend × 18-cap surface is in the generated views.\n\n\u003c!-- Generated from `crispasr --list-backends` + cross-cutting features. --\u003e\n\n| Feature | whisper | parakeet | canary | cohere | granite | granite\u0026#8209;4.1 | voxtral | voxtral4b | qwen3 | fc\u0026#8209;ctc | wav2vec2 | glm\u0026#8209;asr | kyutai\u0026#8209;stt | firered | moonshine | moon\u0026#8209;stream | omniasr | omniasr\u0026#8209;llm | vibevoice | gemma4\u0026#8209;e2b | mimo\u0026#8209;asr | funasr | paraformer | sensevoice |\n|---|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n| Native timestamps | ✔ | ✔ | ✔ | ✔ | | | | | | | | | ✔ | | | | | | | | | | | |\n| CTC timestamps | | | ✔ | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| Word-level timing | ✔ | ✔ | ✔ | ✔ | `-am` | ✔† | `-am` | `-am` | `-am` | `-am` | `-am` | `-am` | ✔ | `-am` | `-am` | `-am` | `-am` | `-am` | | `-am` | `-am` | `-am` | | `-am` |\n| Per-token confidence | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | | ✔ | ✔ | | |\n| Language auto-detect | ✔ | ✔ | LID | LID | LID | LID | LID | LID | ✔ | LID | LID | ✔ | LID | LID | LID | LID | LID | LID | LID | ✔ | LID | LID | LID | ✔ |\n| Speech translation | ✔ | | ✔ | | ✔ | ✔ | ✔ | | ✔ | | | | | | | | | | | | | | | |\n| Speaker diarization | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| Grammar (GBNF) | ✔ | | | | | | | | | | | | | | | | | | | | | | | |\n| Temperature sampling | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | | ✔ | ✔ | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | |\n| Beam search | ✔ | | | | ✔ | ✔ | ✔ | | ✔ | | | ✔ | ✔ | ✔ | ✔ | | ✔ | ✔ | | | | | | |\n| Flash attention | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| Punctuation toggle | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | | ✔ | ✔ | | ✔ | | ✔ | ✔ | | | | ✔ | | ✔ |\n| Punc restoration | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp | pp |\n| Source / target language | | | ✔ | | ✔ | ✔ | ✔ | | ✔ | | | | | | | | | | | | | | | |\n| Audio Q\u0026A (`--ask`) | | | | | * | * | ✔ | | * | | | * | | | | | | | | * | * | | | |\n| Streaming | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| Auto-download (`-m auto`) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| KV quant (`CRISPASR_KV_QUANT`, plus per-half `_K` / `_V`) | | | | | ✔ | ✔ | ✔ | ✔ | ✔ | | | ✔ | | | | | | ✔ | | ✔ | ✔ | ✔ | | |\n| mmap weights (`CRISPASR_GGUF_MMAP`) | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |\n| TTS | | | | | | | | | | | | | | | | | | | ✔ | | | | | |\n\nThe matrix above covers 24 ASR backends. **Additional ASR backends** not shown: `nemotron` (39-lang streaming ASR with cache-aware FastConformer + RNN-T), `lfm2-audio` (ASR + TTS + S2S in one model), `moss-audio` (audio understanding + ASR), `moss-transcribe` (Qwen3-Omni encoder + Qwen3-1.7B ASR), `mini-omni2` (ASR + TTS + S2S), `kugelaudio` (7B audio understanding). See [`docs/feature-matrix.md`](docs/feature-matrix.md) for the full 85-backend matrix. **TTS-only backends** (`kokoro`, `qwen3-tts` + variants, `vibevoice-tts`, `orpheus` + DE variants, `chatterbox` / `chatterbox-turbo` / `kartoffelbox-turbo` / `lahgtna-chatterbox`, `dia`, `bark`, `outetts`, `zonos`, `csm`, `f5-tts`, `parler-tts`, `speecht5`, `piper`, `fastpitch`, `pocket-tts`, `melotts`, `cosyvoice3`, `voxcpm2`, `tada-tts`) all carry the TTS, AUTO_DOWNLOAD, TEMPERATURE, and FLASH_ATTN caps; per-backend cloning + voice-pack support is documented in the [Text-to-Speech models](#text-to-speech-models) table above and [`docs/tts.md`](docs/tts.md). The vibevoice and lfm2-audio columns mark dual-mode (ASR + TTS) backends.\n\n**Key:** ✔ = native/built-in, `-am` = via CTC forced aligner (`-am canary-ctc-aligner.gguf` or `-am qwen3-forced-aligner.gguf`), **LID** = via external language identification pre-step (`-l auto`), **pp** = via `--punc-model` post-processor (FireRedPunc or fullstop-punc), * = experimental or partial support, † = PLUS variant only (native `[T:N]` word timestamps with `-owts`; base uses `-am`). granite-4.1 covers both the regular and `-plus` variants; granite-4.1-nar is a non-autoregressive variant with encoder+projector only (no LLM decode features). The **KV quant** row marks backends that honor `CRISPASR_KV_QUANT={f16,q8_0,q4_0}` — CTC-style backends without a KV cache (parakeet, fc-ctc, wav2vec2, kyutai-stt, firered, moonshine variants, omniasr-CTC) don't apply. The same backends also honor the per-half `CRISPASR_KV_QUANT_K` / `CRISPASR_KV_QUANT_V` overrides (llama.cpp `--cache-type-k` / `--cache-type-v` parity) for asymmetric K-vs-V precision; common recipe `K=q8_0 V=q4_0` saves ~40 % more KV memory than symmetric Q8_0. The **mmap weights** row marks backends consuming `core_gguf::load_weights()` and therefore honoring `CRISPASR_GGUF_MMAP=1`; whisper itself uses upstream's loader and is unaffected. See [`docs/cli.md`](docs/cli.md) Memory footprint for usage + recommended combos.\n\n**Speaker diarization** as a post-processing step via `--diarize`:\n- `energy` / `xcorr` — stereo-only, no extra deps\n- `pyannote` — native GGUF (no Python, no sherpa-onnx); add `--diarize-embedder auto` (TitaNet) or `--diarize-embedder indextts` (ECAPA-TDNN) for globally stable speaker IDs across long files\n- `sherpa` / `ecapa` — external [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) subprocess; runs once globally on full audio for consistent speaker IDs (#110)\n- `vad-turns` — mono-friendly gap-based proxy\n\nThe server endpoint supports `response_format=diarized_json` for structured speaker-labelled output with normalised speaker letters (A, B, C …) — see [`docs/server.md`](docs/server.md#diarized-json-format-206).\n\nFull reference + tuning knobs (cluster threshold, max speakers, pluggable embedder adapters): see [`docs/cli.md#diarization`](docs/cli.md#diarization).\n\n**Language identification** for backends without native LID: `--lid-backend whisper` (default, 75 MB ggml-tiny.bin), `--lid-backend silero` (native GGUF, 16 MB, 95 languages), or `--lid-backend firered` (FireRedLID, 1.7 GB, 120 languages — Conformer encoder + Transformer decoder).\n\n**Voice activity detection**: `--vad` uses the default Silero VAD (~885 KB, auto-downloaded). Each VAD segment is transcribed independently, producing separate SRT/VTT entries with correct timestamps. Use `--vad --split-on-punct` for best subtitle output. Four VAD backends: Silero (default), FireRedVAD (`-vm firered`, recommended), MarbleNet (`-vm marblenet`, 439 KB, 6 languages), Whisper-VAD-EncDec (`-vm whisper-vad`, experimental).\n\n**Punctuation restoration** (`--punc-model`): CTC-based backends output lowercase without punctuation. Named shortcuts: `auto`/`firered` (Chinese+English), `fullstop` (EN/DE/FR/IT, XLM-R-large), `punctuate-all` (12 languages, XLM-R-base), `pcs` (47 languages, punc + truecasing + sentence boundary detection in one model). Or pass a GGUF path directly. Also available via Python/Rust/Dart wrappers (`crispasr.PuncModel`).\n\n**Truecasing** (`--truecase-model`): Restore German noun/name capitalization in lowercase ASR output. Three options in ascending quality: `auto` (statistical, 9 MB), `crf` (CRF with context, 24 MB), `lstm` (BiLSTM char-level, 3.2 MB, **recommended** — 97.9% F1, handles adjective/noun distinction and formal \"Ihnen\"). All auto-download from [`cstr/truecaser-de`](https://huggingface.co/cstr/truecaser-de). Or use `--punc-model pcs` for neural punc + truecasing in one pass (47 languages).\n\n\u003cdetails\u003e\n\u003csummary\u003eWhich backends produce punctuation natively?\u003c/summary\u003e\n\n| Backend | Punctuation | Capitalization | Notes |\n|---|:-:|:-:|---|\n| whisper | ✔ | ✔ | Full punctuation and casing |\n| parakeet | ✔ | ✔ | |\n| canary | ✔ | ✔ | |\n| cohere | ✔ | ✔ | Toggleable via `--no-punctuation` |\n| granite | ✔ | ✔ | LLM output |\n| voxtral | ✔ | ✔ | LLM output |\n| voxtral4b | ✔ | ✔ | LLM output |\n| qwen3 | ✔ | ✔ | LLM output |\n| funasr | ✔ | ✔ | LLM output (Qwen3-0.6B decoder). Chinese chars carry full-width period; mlt-nano variant adds Latin-script casing + punctuation. |\n| sensevoice | ✔ | ✔ | CTC output with native ITN — toggle via `--punctuation` / `--no-punctuation`, controls Arabic-digit vs spelled-out numerals + comma/period emission. |\n| paraformer | **no** | **no** | NAR character-level output — add `--punc-model` |\n| glm-asr | ✔ | ✔ | LLM output |\n| kyutai-stt | ✔ | ✔ | LLM output |\n| moonshine | ✔ | ✔ | Encoder-decoder output |\n| **fastconformer-ctc** | **no** | **no** | CTC — add `--punc-model` |\n| **wav2vec2** | **no** | **no** | CTC — add `--punc-model` |\n| **firered-asr** | **no** | **no** | CTC — add `--punc-model` |\n| **omniasr** (CTC) | **no** | **no** | CTC — add `--punc-model` |\n| **omniasr** (LLM) | ✔ | ✔ | Autoregressive decoder |\n\nOther freely-licensed alternatives that could be added: [felflare/bert-restore-punctuation](https://huggingface.co/felflare/bert-restore-punctuation) (MIT, English, includes truecasing), [xashru/punctuation-restoration](https://github.com/xashru/punctuation-restoration) (Apache-2.0, 40+ languages, BiLSTM-CRF).\n\n\u003c/details\u003e\n\n**Progressive subtitle output** (`--flush-after`): By default, non-whisper backends buffer all segments and print output at the end. For real-time subtitle consumption (PotPlayer, custom media players), use `--flush-after 1` to print each SRT entry to stdout immediately after its VAD segment is transcribed:\n\n```bash\ncrispasr --backend parakeet -m parakeet.gguf --vad --flush-after 1 -osrt -f long_audio.wav\n# SRT entries appear progressively as each segment finishes\n```\n\n**JSON output with language detection**: When using `-l auto -oj`, the JSON output includes detected language info:\n```json\n{\n  \"crispasr\": {\n    \"backend\": \"cohere\",\n    \"language\": \"en\",\n    \"language_detected\": \"en\",\n    \"language_confidence\": 0.977,\n    \"language_source\": \"ecapa\"\n  },\n  \"transcription\": [...]\n}\n```\n\n### Which backend should I pick?\n\n| Need | Pick |\n|---|---|\n| Battle-tested, all features exposed | **whisper** |\n| Lowest English WER | **cohere** |\n| **Fastest** (16x realtime on CPU) | **moonshine** (tiny), **fc-ctc** (10x) |\n| Multilingual + word timestamps + fast | **parakeet** (2.9x RT) |\n| Multilingual with **explicit language control** | **canary** |\n| **Speech translation** (X→en or en→X) | **canary**, **voxtral**, **qwen3** |\n| **30 languages + Chinese dialects** | **qwen3** |\n| **1600+ languages** | **omniasr** (CTC or LLM) |\n| **Realtime streaming ASR** (native incremental encoder, ~2× RT feed; sub-second-token target deferred to phase 2) | **voxtral4b** |\n| Highest-quality offline speech-LLM | **voxtral** |\n| Apache-licensed speech-LLM | **granite**, **voxtral**, **qwen3**, **omniasr-llm** |\n| **Lightweight CTC-only** (fast, no decoder) | **wav2vec2**, **fc-ctc**, **data2vec**, **omniasr** |\n| **Mandarin + Chinese dialects** | **firered-asr**, **qwen3**, **glm-asr**, **funasr**, **paraformer**, **sensevoice** |\n| **Multilingual (31 langs) speech-LLM** | **fun-asr-mlt-nano**, **qwen3**, **omniasr-llm**, **gemma4-e2b** |\n| **Multilingual (50+ langs) + LID + emotion + audio-event in one pass** | **sensevoice** (encoder-only CTC, non-AR, 15× faster than Whisper-Large) |\n\n### CPU performance tips\n\nAudio-LLM backends (`qwen3`, `voxtral`, `granite`, `glm-asr`, etc.) run full\ntransformer decoder stacks (28+ layers, 2048-dim) and are **dramatically slower\non CPU** than encoder-only backends. On older dual-core hardware they can drop\nbelow 0.01× realtime. If you're on CPU-only hardware:\n\n- Prefer **moonshine** (16× RT), **fc-ctc** (10× RT), **parakeet** (2.9× RT),\n  or **whisper** for usable speeds.\n- Use `--flush-after 1` to see results as each VAD slice completes instead of\n  waiting for the entire file.\n- Use `-pp` / `--print-progress` for per-slice progress indicators on all\n  backends (unified backends show slice-level progress; whisper shows\n  encoder-level progress).\n- Quantize models to Q4_K or Q5_K to reduce memory and compute.\n\n### Language detection for backends that don't do it natively\n\nCohere, canary, granite, voxtral and voxtral4b need an explicit\nlanguage code up front. If you don't know the language, pass\n`-l auto` and crispasr runs an optional LID pre-step before the main\ntranscribe() call:\n\n```bash\n# Downloads ggml-tiny.bin (75 MB, 99 languages) on first use\ncrispasr --backend cohere -m $TC/cohere-transcribe-q5_0.gguf \\\n         -f unknown.wav -l auto\n# crispasr[lid]: detected 'en' (p=0.977) via whisper-tiny\n# crispasr: LID -\u003e language = 'en' (whisper, p=0.977)\n```\n\nThese LID providers are available:\n\n- `--lid-backend whisper` (default) — uses a small multilingual ggml-*.bin model via the crispasr C API. Auto-downloads ~75 MB on first use. 99 languages.\n- `--lid-backend silero` — native GGUF port of Silero's 95-language classifier. 16 MB F32, pure C++. Faster and smaller than whisper-tiny but slightly less accurate on long audio (\u003e20s).\n- `--lid-backend ecapa` — **recommended**: ECAPA-TDNN (Apache-2.0). Purpose-built for language ID. Very high accuracy on TTS benchmark. Two variants via `--lid-model`:\n  - [`cstr/ecapa-lid-107-GGUF`](https://huggingface.co/cstr/ecapa-lid-107-GGUF) — VoxLingua107, 43 MB F16, 107 languages, ISO codes (en, de, ...). **Default.**\n  - [`cstr/ecapa-lid-commonlanguage-GGUF`](https://huggingface.co/cstr/ecapa-lid-commonlanguage-GGUF) — CommonLanguage, 40 MB F16, 45 languages, full names (English, German, ...).\n- `--lid-backend firered` — FireRedLID (Conformer encoder + Transformer decoder). Q4_K (544 MB), 120 languages including Chinese dialects. Slower but covers more languages.\n\nThese VAD providers are available:\n\n- **Silero VAD** (default) — ~885 KB, auto-downloaded via `--vad`. Industry-standard, well-tested.\n- **FireRedVAD** — DFSMN-based, 2.4 MB, F1=97.57%. Pass `--vad -vm firered` to auto-download. Recommended.\n- **MarbleNet** — NVIDIA 1D separable CNN, 439 KB, 6 languages (EN/DE/FR/ES/RU/ZH). Pass `--vad -vm marblenet` to auto-download. Smallest model. ([`cstr/marblenet-vad-GGUF`](https://huggingface.co/cstr/marblenet-vad-GGUF))\n- **Whisper-VAD-EncDec** *(experimental)* — Whisper-base encoder + TransformerDecoder head, 22 MB Q4_K. Trained on Japanese ASMR; may not generalise well to all domains. Pass `--vad -vm whisper-vad`. Slower than others (~1s vs ~50ms). ([`cstr/whisper-vad-encdec-asmr-GGUF`](https://huggingface.co/cstr/whisper-vad-encdec-asmr-GGUF))\n\nPass `--lid-backend off` to skip LID entirely.\n\n### Text language identification (post-ASR / standalone)\n\nAudio LID (above) tags **what was spoken**; text LID tags **what was\nwritten**. Text LID runs on a transcript or any UTF-8 string and is\nuseful for routing post-ASR pipelines (translation, punctuation, sub\nselection) without re-running an audio model. Three GGUF families,\none binary — the dispatcher picks by `general.architecture`:\n\n| Backend | Labels | Size (F16) | License | HF repo |\n|---|---:|---:|---|---|\n| **CLD3** (Google compact language detector v3) | 109 ISO 639-1 | **440 KB** | Apache-2.0 | [`cstr/cld3-GGUF`](https://huggingface.co/cstr/cld3-GGUF) |\n| **GlotLID-V3** (cis-lmu fastText) | 2102 ISO 639-3 + script | 250 MB | Apache-2.0 | [`cstr/glotlid-GGUF`](https://huggingface.co/cstr/glotlid-GGUF) |\n| **LID-176** (Facebook fastText) | 176 ISO 639-1 | 63 MB | CC-BY-SA-3.0¹ | [`cstr/fasttext-lid176-GGUF`](https://huggingface.co/cstr/fasttext-lid176-GGUF) |\n\n¹ LID-176 is **CC-BY-SA-3.0 (viral)** — redistributors of the GGUF\ninherit ShareAlike. CLD3 + GlotLID-V3 are Apache-2.0 with no such\nconstraint. Pick CLD3 for the smallest, fastest path; GlotLID for\nmaximum coverage (low-resource languages); LID-176 only if you need\nits specific 176-label space and accept the SA obligation.\n\n**Standalone CLI** — auto-routes by GGUF arch, with auto-download:\n\n```bash\ncrispasr-lid -m auto --text \"Bonjour le monde\"        # → cstr/cld3-GGUF (default, ~440 KB)\ncrispasr-lid -m auto:glotlid --text \"Bonjour le monde\" -k 5\ncrispasr-lid -m auto:lid-fasttext176 --text \"Hallo Welt\"\n# Or pass an explicit path / canonical filename (looked up in the registry):\ncrispasr-lid -m cld3-f16.gguf --text \"你好世界\"\n# zh\t0.997816\necho \"Привет мир\" | crispasr-lid -m auto --quiet\n# ru\t0.907322\n```\n\n**Post-ASR pipeline** — `--lid-on-transcript` runs the same dispatcher\non the assembled transcript (also accepts `auto[:variant]`):\n\n```bash\ncrispasr -m ggml-tiny.bin -f speech.wav --lid-on-transcript auto\n# (transcript on stdout)\n# lang=de\tconf=0.997123\tbackend=lid-cld3\n```\n\nThe dispatcher (`src/text_lid_dispatch.{h,cpp}`) is a thin C ABI\nfaçade — one integer compare per call; per-stage diff harness is\ngreen at cos≥0.999 across 8 multilingual smoke samples.\n\n---\n\n## Install \u0026 build\n\n```bash\ngit clone https://github.com/CrispStrobe/CrispASR\ncd CrispASR\ncmake -B build -DCMAKE_BUILD_TYPE=Release\ncmake --build build -j$(nproc)\n```\n\nProduces `build/bin/crispasr` (main CLI), `build/bin/crispasr-quantize`,\nand `build/bin/crispasr-diff`. No Python, PyTorch, or pip required at\nruntime — just a C++17 compiler and CMake 3.14+.\n\nFor GPU acceleration, add the matching ggml flag at configure time:\n\n```bash\ncmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON     # NVIDIA\ncmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON    # Apple Silicon\ncmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON   # cross-vendor\n```\n\n**See [`docs/install.md`](docs/install.md)** for the full guide:\nall GPU backends (CUDA / Metal / Vulkan / MUSA / SYCL), Windows\nconvenience scripts, ffmpeg ingestion, optional BLAS, glibc notes,\nand the `scripts/dev-build.sh` wrapper.\n\n---\n\n## Quick start\n\nASR examples below; for TTS see the [Text-to-Speech](#text-to-speech-tts) section.\n\n### Whisper (historical path, byte-identical to upstream whisper.cpp)\n\n```bash\n# Download a whisper model (same as upstream whisper.cpp)\n./models/download-ggml-model.sh base.en\n\n./build/bin/crispasr -m models/ggml-base.en.bin -f samples/jfk.wav\n# [00:00:00.000 --\u003e 00:00:07.940]   And so my fellow Americans ask not what your country can do for you\n# [00:00:07.940 --\u003e 00:00:10.760]   ask what you can do for your country.\n```\n\n### Parakeet (multilingual, free word timestamps, fastest)\n\n```bash\n# Grab the quantized model (~467 MB)\ncurl -L -o parakeet.gguf \\\n    https://huggingface.co/cstr/parakeet-tdt-0.6b-v3-GGUF/resolve/main/parakeet-tdt-0.6b-v3-q4_k.gguf\n\n./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav\n# Auto-detected backend 'parakeet' from GGUF metadata.\n# And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.\n\n# Word-level timestamps (one line per word)\n./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav -ml 1\n```\n\n### Canary (explicit language, speech translation)\n\n```bash\n# Transcription (source == target)\n./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl de\n\n# Translation (German speech → English text)\n./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl en\n\n# ...or use the familiar crispasr flag:\n./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -l de --translate\n```\n\n### Voxtral (speech-LLM with auto-download)\n\n```bash\n# First run downloads ~2.5 GB to ~/.cache/crispasr/ via curl, then runs\n./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav\n\n# Subsequent runs use the cached file\n./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav -l en\n```\n\n### Qwen3-ASR (30 languages + Chinese dialects)\n\n```bash\n# 0.6B (default, ~500 MB)\n./build/bin/crispasr --backend qwen3 -m auto -f audio.zh.wav\n\n# 1.7B (higher quality, ~1.3 GB) — supports both -hf and non-hf source models\n./build/bin/crispasr --backend qwen3 -m qwen3-1.7b --auto-download -f audio.wav\n\n# Japanese anime/galgame fine-tune (~1.3 GB)\n./build/bin/crispasr --backend qwen3 -m qwen3-ja-anime --auto-download -f anime.wav\n```\n\n### MiMo-V2.5-ASR (Mandarin + dialects + English, 7.5B Qwen2 LM)\n\n```bash\n# Download the LM + audio tokenizer (the tokenizer is a separate model)\nhuggingface-cli download cstr/mimo-asr-GGUF mimo-asr-q4_k.gguf \\\n    --local-dir ~/.cache/crispasr\nhuggingface-cli download cstr/mimo-tokenizer-GGUF mimo-tokenizer-q4_k.gguf \\\n    --local-dir ~/.cache/crispasr\n\n# Transcribe (auto-discovers tokenizer if it sits next to the LM)\n./build/bin/crispasr \\\n    --backend mimo-asr \\\n    -m ~/.cache/crispasr/mimo-asr-q4_k.gguf \\\n    --codec-model ~/.cache/crispasr/mimo-tokenizer-q4_k.gguf \\\n    -f samples/jfk.wav\n# Output: And so, my fellow Americans, ask not what your country can do\n# for you. Ask what you can do for your country.\n```\n\nThe 4.5 GB Q4_K is the recommended quant; F16 (14.9 GB) needs ~16 GB\nRAM during inference. JFK matches the upstream Python\n`MimoAudio.asr_sft` reference verbatim; performance on M1+Metal is\n~0.3× realtime (Q4_K dequant per step is the bottleneck — F16 +\nKV-reuse follow-ups are queued under PLAN #51a/b/c).\n\n### Wav2Vec2 (lightweight CTC, any HF Wav2Vec2ForCTC model)\n\n```bash\n# English (Q4_K quantized, 212 MB — 6x smaller than F16)\ncurl -L -o wav2vec2-en-q4k.gguf \\\n    https://huggingface.co/cstr/wav2vec2-large-xlsr-53-english-GGUF/resolve/main/wav2vec2-xlsr-en-q4_k.gguf\n\n./build/bin/crispasr -m wav2vec2-en-q4k.gguf -f samples/jfk.wav\n# and so my fellow americans ask not what your country can do for you ask what you can do for your country\n\n# German\ncurl -L -o wav2vec2-de-q4k.gguf \\\n    https://huggingface.co/cstr/wav2vec2-large-xlsr-53-german-GGUF/resolve/main/wav2vec2-xlsr-de-q4_k.gguf\n\n./build/bin/crispasr -m wav2vec2-de-q4k.gguf -f audio.de.wav\n\n# Convert any HuggingFace Wav2Vec2ForCTC model:\npython models/convert-wav2vec2-to-gguf.py \\\n    --model-dir jonatasgrosman/wav2vec2-large-xlsr-53-german \\\n    --output wav2vec2-de.gguf --dtype f32\n# Then optionally quantize:\n./build/bin/crispasr-quantize wav2vec2-de.gguf wav2vec2-de-q4k.gguf q4_k\n```\n\n---\n\n## Streaming, TTS, and HTTP server\n\nCrispASR has three feature areas that warrant their own docs pages:\n\n- **[Streaming \u0026 live transcription](docs/streaming.md)** — `--stream`,\n  `--mic`, `--live`, sliding-window chunking, per-token confidence.\n- **[Text-to-Speech (TTS)](docs/tts.md)** — Kokoro (multilingual,\n  smallest), Qwen3-TTS (highest fidelity, voice cloning), VibeVoice\n  (lowest-latency streaming), Orpheus (3 B Llama + SNAC), Chatterbox\n  (flow-matching + HiFT vocoder, German via Kartoffelbox), IndexTTS,\n  VoxCPM2, and CosyVoice3 (9 langs + 18 zh dialects; baked-voice bank +\n  arbitrary-WAV cloning). Voice packs, language routing, and qwen3-tts\n  environment switches. All TTS output is watermarked; post-embed\n  verification warns if confidence is low. Use\n  `--detect-watermark file.wav` to check any WAV for AI watermarks.\n- **[Server mode (HTTP API)](docs/server.md)** — persistent model,\n  OpenAI-compatible `/v1/audio/transcriptions` (ASR) and\n  `/v1/audio/speech` + `/v1/voices` (TTS, automatic on any loaded\n  CAP_TTS backend), per-request voice + speed + instructions, CORS,\n  long-form sentence chunking, API keys, Docker Compose, prebuilt\n  CUDA images.\n\nQuickest taste of each:\n\n```bash\n# Streaming from microphone\ncrispasr --mic -m model.gguf\n\n# TTS via auto-downloaded VibeVoice (~636 MB on first run)\ncrispasr --backend vibevoice-tts -m auto --tts \"Hello world\" --tts-output hello.wav\n\n# CosyVoice3 on GPU; companions auto-download beside the LLM\ncrispasr --backend cosyvoice3-tts -m auto --tts \"Hello world\" --tts-output cosy.wav\n\n# CosyVoice3 fast mode: 5 flow steps instead of the quality-default 10\nCOSYVOICE3_FLOW_STEPS=5 crispasr --backend cosyvoice3-tts -m auto \\\n  --tts \"Hello world\" --tts-output cosy-fast.wav\n\n# Persistent HTTP server, OpenAI-compatible\ncrispasr --server -m model.gguf --port 8080\ncurl -F \"file=@audio.wav\" http://localhost:8080/v1/audio/transcriptions\n\n# TTS over HTTP — load a TTS backend, hit /v1/audio/speech\ncrispasr --server --backend qwen3-tts-customvoice -m auto --voice-dir ./voices --port 8080\ncurl http://localhost:8080/v1/audio/speech \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"input\":\"Hello world\",\"voice\":\"vivian\"}' -o out.wav\n```\n\nCosyVoice3 uses batched classifier-free guidance and request-sized KV\ncaching by default. Baked voices load only the LLM, flow, HiFT, and voice\nbank; the larger S3 tokenizer and CAMPPlus companions load lazily when a\n`.wav` cloning voice is first requested.\n\n---\n\n## CLI reference\n\nCommon flags:\n\n```bash\ncrispasr -m auto --backend parakeet -f audio.wav --vad -osrt --split-on-punct\n```\n\n| Flag | Meaning |\n|---|---|\n| `-m FNAME` / `--backend NAME` | Model path (or `auto`) and forced backend |\n| `-f FNAME` | Input audio (repeatable; positional accepted) |\n| `--vad` | Silero VAD chunking — strongly recommended for multi-minute audio |\n| `-osrt` / `-ovtt` / `-otxt` / `-oj` / `-ojf` | Output formats (also `-ocsv`, `-olrc`) |\n| `-am FNAME` | CTC aligner GGUF for word-level timestamps on LLM backends |\n| `--align-only` | Standalone forced alignment: text/`.srt` + audio → timestamped SRT/JSON (no ASR needed); `.srt` input keeps its cues and gets re-timed (`--align-granularity auto\\|word\\|segment`) |\n| `-tp F` / `-bs N` | Sampling temperature / beam search width |\n| `-n N` / `--frequency-penalty F` | Generated-token cap / opt-in repeated-token penalty for supported autoregressive ASR backends |\n| `-l auto` / `--detect-language` | LID pre-step for backends without native lang detect |\n| `--hotwords \"A,B,C\"` | Contextual biasing — boost named terms during CTC/TDT decode or LLM prompt |\n| `-ck N` | Fallback chunk size when VAD is off (default 30 s) |\n| `--list-backends` | Print the capability matrix and exit |\n\n**See [`docs/cli.md`](docs/cli.md)** for the full reference: every\nflag, VAD details, CTC alignment workflow, output JSON layout, the\nauto-download registry, and supported audio formats. **See\n[`docs/bindings.md`](docs/bindings.md)** for Python / Rust / Dart /\nGo / Java / JavaScript / Ruby / mobile.\n\n---\n\n## Architecture, contributing, regression matrix\n\nCrispASR is structured as a stable C-ABI in `src/` (every algorithm:\nVAD, diarize, LID, alignment, cache, registry) consumed by all\nlanguage wrappers, with thin presentation layers in `examples/cli/`.\nPer-model runtimes live in `src/{whisper,parakeet,canary,...}.cpp`,\nsharing primitives from `src/core/` (mel, ffn, attention, GGUF\nloader, FastConformer / Conformer / Granite-LLM blocks, etc.).\n\n- **[`docs/architecture.md`](docs/architecture.md)** — full layered\n  layout, file-by-file tour of `src/` and `examples/cli/`,\n  per-backend internals table, regression discipline.\n- **[`docs/contributing.md`](docs/contributing.md)** — adding a new\n  backend in five files, clang-format-18 setup, the\n  `crispasr-diff` PyTorch-ground-truth workflow, and the\n  TTS audio-cosine-vs-reference regression target.\n- **[`docs/regression-matrix.md`](docs/regression-matrix.md)** —\n  `tools/test-all-backends.py` capability tiers, cache modes\n  (`keep` / `ephemeral`), `--skip-missing` for CI.\n\n**Shared libraries** (cross-repo with CrispEmbed):\n- `crisp_audio/` — Whisper-shape audio encoder (Conv-stem + Transformer)\n- `crisp_punc/` — punctuation restoration (FireRedPunc + PCS)\n- `crisp_lid/` — text-based language identification (fastText + CLD3)\n- `crisp_truecase/` — truecasing (statistical + CRF + BiLSTM)\n\nBoth are self-contained static libraries with CMakeLists.txt. CrispEmbed\nlinks them via `add_subdirectory(../CrispASR/crisp_*/)`; CrispASR uses\nthem directly. If the shared dir is absent, both repos fall back to local\ncopies of the source files.\n\nFor benchmarks see [`PERFORMANCE.md`](PERFORMANCE.md); for the\nsession-by-session port log and the bug-class lessons, see\n[`LEARNINGS.md`](LEARNINGS.md).\n\n---\n\n## Quantize models\n\n`build/bin/crispasr-quantize` is a single, model-agnostic GGUF\nre-quantization tool that works across all supported model families\n(Whisper, Parakeet, Canary, Cohere, Voxtral, Qwen3, Granite, Wav2Vec2,\nMiMo-ASR, GLM-ASR, Moonshine, VibeVoice, Kokoro, Qwen3-TTS, …):\n\n```bash\n./build/bin/crispasr-quantize input.gguf output.gguf q4_k\n```\n\n**See [`docs/quantize.md`](docs/quantize.md)** for the full guide:\nsupported quant types, K-quant alignment fallback, recommended quant\nper backend, and worked examples for each architecture.\n\n---\n\n## GPU backend selection\n\nAll backends use `ggml_backend_init_best()` which automatically picks the highest-priority compiled backend: CUDA \u003e Metal \u003e Vulkan \u003e CPU. To force a specific backend:\n\n```bash\n# Force Vulkan even when CUDA is available\ncrispasr --gpu-backend vulkan -m model.gguf -f audio.wav\n\n# Pin a specific GPU (useful on Vulkan systems with iGPU + dGPU)\ncrispasr --gpu-backend vulkan -dev 1 -m model.gguf -f audio.wav\n\n# Force CPU (useful for benchmarking)\ncrispasr -ng -m model.gguf -f audio.wav\n\n# CUDA unified memory (swap to RAM when VRAM exhausted)\nGGML_CUDA_ENABLE_UNIFIED_MEMORY=1 crispasr -m model.gguf -f audio.wav\n```\n\nBuild flags: `-DGGML_CUDA=ON`, `-DGGML_METAL=ON`, `-DGGML_VULKAN=ON`.\n\nNotes:\n- `--gpu-backend vulkan` selects the Vulkan backend, but it does not choose which physical GPU to use. Use `-dev N` to select the Vulkan device index.\n- On some Windows laptops, Vulkan device `0` is the Intel iGPU and the NVIDIA GPU is `1`. If Vulkan looks unexpectedly slow, rerun with `-dev 1`.\n- The Windows convenience script `build-vulkan.bat` creates a separate Vulkan-capable binary at `build-vulkan\\bin\\crispasr.exe`.\n\n---\n\n## Debugging \u0026 profiling\n\nFor most backends, `-v` / `--verbose` surfaces per-stage timings and\ndevice picks. For headless / library use (where the CLI flag isn't\nplumbed through), set `CRISPASR_VERBOSE=1` instead.\n\n```bash\n# Per-stage timing breakdown (mel / encoder / prefill / decode):\ncrispasr -v --backend gemma4-e2b -m model.gguf -f audio.wav\n# gemma4_e2b: mel 128x1099 (17.2 ms)\n# gemma4_e2b: encoder done: 1536x275 (719.0 ms)\n# gemma4_e2b: prefill done, first_token=3133 (1464.0 ms)\n# gemma4_e2b: decoded 25 tokens (7748.3 ms total)\n# crispasr: transcribed 11.0s audio in 7.75s (1.4x realtime)\n\n# Hugging Face access for gated models (Voxtral, Gemma4-E2B, …):\nHF_TOKEN=hf_xxx crispasr -m auto --backend gemma4-e2b -f audio.wav\n```\n\nThe server has its own auth env: `CRISPASR_API_KEYS` (see\n[Server mode](#server-mode-persistent-model-http-api)).\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cb\u003ePer-backend debug / bench / dump-dir env vars (developer)\u003c/b\u003e\u003c/summary\u003e\n\nThese are useful when porting a new backend or chasing a regression.\nThe `*_BENCH=1` toggles emit per-stage timings even without `-v`; the\n`*_DEBUG=1` toggles emit per-step diagnostic prints; the `*_DUMP_DIR=`\npaths write per-stage F32 tensors for diff-testing against a PyTorch\nreference (see [Debug a new backend against PyTorch ground truth](#debug-a-new-backend-against-pytorch-ground-truth)).\n\n| Env var | Purpose |\n| --- | --- |\n| `CRISPASR_VERBOSE=1` | Forces verbose mode for any backend (parallel to the `-v` flag). |\n| `CRISPASR_DUMP_DIR=path/` | Generic per-stage F32 tensor dump for the `crispasr-diff` harness. |\n| `GEMMA4_E2B_BENCH=1` | Per-stage timings for the Gemma-4-E2B backend. |\n| `COHERE_BENCH=1` / `COHERE_DEBUG=1` | Cohere transcribe per-stage timings / per-step diagnostics. |\n| `COHERE_PROF=1` | Cohere graph-level profiling (per-op timings). |\n| `COHERE_THREADS=N` | Override thread count for the Cohere backend. |\n| `COHERE_DEVICE=cpu\\|cuda\\|metal\\|vulkan` | Force the Cohere backend onto a specific device. |\n| `COHERE_DUMP_ATTN=path/` | Dump attention activations for Cohere (used by the diff harness). |\n| `FIRERED_BENCH=1` | Per-stage timings for the FireRedASR backend. |\n| `FIREREDPUNC_DEBUG=1` | Per-step diagnostics for the FireRed punctuation post-step. |\n| `MOONSHINE_STREAMING_BENCH=1` | Per-stage timings for moonshine-streaming. |\n| `OMNIASR_BENCH=1` / `OMNIASR_DEBUG=1` / `OMNIASR_DUMP_DIR=` | OmniASR per-stage timings, diagnostics, and stage dumps. |\n| `PARAKEET_DEBUG=1` | Parakeet TDT per-step diagnostics (joint network, blank-id sanity). |\n| `QWEN3_TTS_BENCH=1` / `QWEN3_TTS_DEBUG=1` / `QWEN3_TTS_DUMP_DIR=` | Qwen3-TTS per-stage timings, diagnostics, and stage dumps. |\n| `VIBEVOICE_BENCH=1` / `VIBEVOICE_DEBUG=1` / `VIBEVOICE_DUMP_DIR=` | VibeVoice ASR per-stage timings, diagnostics, and stage dumps. |\n| `VIBEVOICE_REF_FEATURES=path` | Replace the live encoder with a saved feature tensor (regression harness). |\n| `VIBEVOICE_TTS_DUMP=path/` | VibeVoice TTS per-stage dumps (token IDs, base/TTS hidden, neg condition, frame-0 noise/v_cfg/latent/acoustic_embed) for the diff harness. |\n| `VIBEVOICE_TTS_DUMP_PERFRAME=1` | Per-frame VibeVoice TTS dumps written as `perframe_\u003cstage\u003e_f\u003cNNN\u003e.bin`. Pair with `VIBEVOICE_TTS_DUMP=path/` and `VIBEVOICE_TTS_NOISE=path` for stage-by-stage AR diff against `tools/run_official_vibevoice.py`. |\n| `VIBEVOICE_TTS_TRACE=1` | Extra one-line traces (negative-condition prefill rms, scaling/bias factors loaded). Same effect as `-vv`. |\n| `VIBEVOICE_VOICE_AUDIO=path.wav` | Reference voice WAV for 1.5B-base TTS without a `.gguf` voice cache. |\n| `VIBEVOICE_TTS_NOISE=path` | Override the per-frame Gaussian init noise. Flat little-endian float32 `[N_frames, vae_dim]` — typically the `noise.bin` written by `tools/run_official_vibevoice.py`. |\n| `VIBEVOICE_VAE_BACKEND=cpu\\|metal\\|cuda\\|vulkan` | Pin the VAE decoder onto a specific backend. |\n| `WAV2VEC2_BENCH=1` / `WAV2VEC2_VERBOSE=1` / `WAV2VEC2_DUMP_DIR=` | wav2vec2 per-stage timings, verbose graph traces, and stage dumps. |\n| `CRISPASR_VOXTRAL4B_STREAM_TIMING=1` | Per-stage timings for the voxtral4b streaming path (encoder drain / prefill / first-text-token / decode-step p50/p95). |\n| `CRISPASR_VOXTRAL4B_STREAM_CHUNK_MS=N` | Override the internal encoder chunk size (default 240 ms). Must be a multiple of 80 ms. Larger = faster feed (kernel-launch amortisation), longer live-caption latency floor. |\n| `CRISPASR_VOXTRAL4B_STREAM_BATCH_ENCODER=1` | Regression-debug: ignore the streaming encoder's audio_embeds and re-run the whole batch encoder at flush. |\n| `CRISPASR_VOXTRAL4B_STREAM_DEBUG=1` / `CRISPASR_VOXTRAL4B_STREAM_DIFF=1` | Per-step decode prints / side-by-side encoder cosine vs the batch encoder. |\n| `CRISPASR_VOXTRAL4B_STREAM_LIVE=1` | Live-captions decode-during-feed (PLAN #7 phase 3). `get_text()` polled during feed returns progressive transcript. Default OFF (PTT semantics). Wrappers: Python `Session.stream_open(live=True)`, Rust `stream_open_ex(.., live: true)`. |\n| `CRISPASR_VOXTRAL4B_STREAM_DECODER_THREAD=1` | Decoder worker thread (PLAN #7 phase 4, implies live mode). Lets `feed()` return between encoder chunks without waiting for the decode loop — useful for mic-driven workloads. On M1 the Metal queue serializes encoder and decoder so total wall-clock is unchanged; faster GPUs with kernel-level parallelism see real overlap. |\n| `CRISPASR_VOXTRAL4B_FUSED_QKV=0` | Opt out of the runtime fused-QKV LLM path (default-on, ~7-8 % decode speedup on M1 Q4_K, ~500 MB extra memory). |\n| `CRISPASR_QWEN3_ASR_FUSED_QKV=0` | Opt out of the runtime fused-QKV LLM path for qwen3-asr (default-on; works on F16/F32/Q4_K/Q8_0/...). |\n| `CRISPASR_VOXTRAL_FUSED_QKV=1` | Opt **in** to the runtime fused-QKV LLM path for voxtral 3B. Off by default (no measurable speedup on JFK-shape decodes; useful for long-form workloads where decode dominates). |\n| `QWEN3_TTS_FUSED_QKV=1` | Opt in to the runtime fused-QKV talker path. |\n| `GRANITE_DISABLE_ENCODER_GRAPH=1` | Force the granite-speech / -plus / -nar encoder back to the per-layer CPU loop (slower but kept around for debugging). The single ggml-graph encoder with per-layer Shaw RPE is the default and is bit-near-identical to the CPU loop while being ~2× faster end-to-end across all three variants. |\n| `CRISPASR_NO_REL_POS=1` | Ablate the relative-position bias in the Gemma-4 audio encoder (development only). |\n| `ECAPA_REF_FBANK=path` | Reference filterbank tensor for the ECAPA-TDNN LID model (regression harness). |\n| `CRISPASR_SHERPA_LID_BIN=path` | Override the auto-detected sherpa-onnx LID binary. |\n| `CRISPASR_ARG_DEVICE=N` | Default GPU device index when `-dev` isn't passed. |\n| `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` | Let CUDA swap to RAM when VRAM is exhausted. |\n| `GGML_VK_VISIBLE_DEVICES` / `CUDA_VISIBLE_DEVICES` | Standard ggml/CUDA device-visibility filters. |\n\n`HF_TOKEN` and `HUGGING_FACE_HUB_TOKEN` are both honoured for gated-model\ndownloads (in that order).\n\n\u003c/details\u003e\n\n---\n\n## Credits\n\n- **[whisper.cpp](https://github.com/ggml-org/whisper.cpp)** — the original ggml inference engine and Whisper runtime this fork is built on\n- **[ggml](https://github.com/ggml-org/ggml)** — the tensor library everything runs on\n- **NVIDIA NeMo** — parakeet-tdt-{0.6b-v2,0.6b-v3,1.1b}, parakeet-tdt_ctc-{110m,1.1b,0.6b-ja}, parakeet-ctc-{0.6b,1.1b}, canary-1b-v2, canary-ctc aligner, and the FastConformer-CTC family (stt_en_fastconformer_ctc_{large,xlarge,xxlarge})\n- **Cohere** — cohere-transcribe-03-2026\n- **Qwen team (Alibaba)** — Qwen3-ASR-0.6B, Qwen3-ASR-1.7B, Qwen3-ForcedAligner-0.6B\n- **Mistral AI** — Voxtral Mini 3B and 4B Realtime\n- **IBM Granite team** — Granite Speech 3.2-8b, 3.3-2b, 3.3-8b, 4.0-1b\n- **Meta / wav2vec2** — wav2vec2 CTC models (XLSR-53 English, German, multilingual via any Wav2Vec2ForCTC checkpoint)\n- **[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)** — optional diarization via subprocess (ONNX models)\n- **[Silero](https://github.com/snakers4/silero-vad)** — VAD (native GGUF) and language identification (native GGUF, 95 languages)\n- **[pyannote](https://github.com/pyannote/pyannote-audio)** — speaker diarization segmentation (native GGUF port)\n- **[miniaudio](https://miniaud.io/)** + **[stb_vorbis](https://github.com/nothings/stb)** + **[libopus/opusfile](https://opus-codec.org/)** — embedded/linked audio decoders (WAV/MP3/FLAC/AIFF/OGG/Opus, no ffmpeg; AAC/M4A/ALAC via Apple AudioToolbox)\n- **[Claude Code](https://claude.ai/claude-code)** (Anthropic) — significant portions of the crispasr integration layer, all model converters, and the FastConformer/attention/mel/FFN/BPE core helpers were co-authored with Claude\n\n---\n\n## License\n\nSame as upstream whisper.cpp: **MIT**.\n\nPer-model weights are covered by their respective HuggingFace model licenses (see [Supported backends](#supported-backends)). The `crispasr` binary itself links model runtimes that are mostly permissively licensed (MIT / Apache-2.0 / CC-BY-4.0 for weights).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrispstrobe%2Fcrispasr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcrispstrobe%2Fcrispasr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcrispstrobe%2Fcrispasr/lists"}