{"id":51195743,"url":"https://github.com/kellpossible/neural-tts","last_synced_at":"2026-06-27T20:01:55.141Z","repository":{"id":360808331,"uuid":"1251271465","full_name":"kellpossible/neural-tts","owner":"kellpossible","description":"A bridge between operating system TTS interface and various newly available TTS libraries","archived":false,"fork":false,"pushed_at":"2026-05-28T01:04:46.000Z","size":186,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-28T01:21:18.323Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kellpossible.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-27T12:15:04.000Z","updated_at":"2026-05-28T01:04:51.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kellpossible/neural-tts","commit_stats":null,"previous_names":["kellpossible/neural-tts"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/kellpossible/neural-tts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kellpossible%2Fneural-tts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kellpossible%2Fneural-tts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kellpossible%2Fneural-tts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kellpossible%2Fneural-tts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kellpossible","download_url":"https://codeload.github.com/kellpossible/neural-tts/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kellpossible%2Fneural-tts/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34866118,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-27T02:00:06.362Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-27T20:01:54.261Z","updated_at":"2026-06-27T20:01:55.136Z","avatar_url":"https://github.com/kellpossible.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# neural-tts\n\nA bridge between operating system TTS interface and various newly available TTS libraries.\n\nCurrently `speech-dispatcher` on Linux is supported, with five providers:\n\n- **kokoro-onnx** — Kokoro-82M via ONNX runtime. CPU or GPU, dozens of\n  pre-baked voices, multilingual.\n- **longcat-audiodit** — Meituan's [LongCat-AudioDiT](https://github.com/meituan-longcat/LongCat-AudioDiT)\n  1B diffusion model. CUDA-only, zero-shot voice cloning from user-supplied\n  reference clips, English + Chinese.\n- **moss-tts-nano** — OpenMOSS's [MOSS-TTS-Nano](https://github.com/OpenMOSS/MOSS-TTS-Nano)\n  100M autoregressive ONNX model. CPU-friendly (claims realtime on 4 cores),\n  zero-shot voice cloning, 20 languages.\n- **omnivoice** — k2-fsa's [OmniVoice](https://github.com/k2-fsa/OmniVoice)\n  diffusion language model. GPU-recommended (CUDA/XPU/MPS), zero-shot voice\n  cloning, 600+ languages, optional Whisper auto-transcription of reference\n  clips.\n- **qwen3-tts** — Alibaba's [Qwen3-TTS-0.6B](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base)\n  via [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts)\n  (CUDA-graph capture, no flash-attn / vLLM / Triton). CUDA-only, zero-shot\n  voice cloning, native streaming, 10 languages. Optional RTF-aware jitter\n  buffer for sub-realtime hardware.\n\n## Why\n\nKDE Plasma 6 routes TTS through `qt6-qtspeech-speechd` → `speech-dispatcher`.\nThe default voices (espeak-ng) sound robotic. Kokoro produces dramatically\nmore natural audio but loads in seconds, so we need a resident daemon to keep\nthe model warm.\n\n## Architecture\n\n```\nKDE app → QtSpeech → speech-dispatcher → sd_generic → bin/sd-neural-tts\n                                                          │\n                                                  AF_UNIX socket (protobuf framing)\n                                                          │\n                                              neural-tts.service (resident daemon)\n                                                          │\n                                              socketpair, NEURAL_TTS_PROVIDER_FD\n                                                          │\n                                              provider subprocess\n                                              (its own uv venv, model warm)\n```\n\n- **Wire format**: length-prefixed protobuf (`proto/neural_tts.proto`). Audio is\n  streamed as raw PCM in `AudioChunk` messages.\n- **Speechd-side wire rate**: always 24 kHz s16le mono; the daemon resamples\n  on the fly via `soxr` if the active provider's native rate differs.\n- **Provider isolation**: each provider is its own uv project with its own\n  `.venv/`. Only one provider subprocess is alive at a time.\n- **Auto-routing**: the daemon keeps an in-memory voice index mapping every\n  voice id to its owning provider. The first LIST VOICES from speechd populates\n  it by spawning each enabled provider in *lazy* mode (no model load) to\n  enumerate voices, then shutting it down. On synth, the daemon looks up the\n  voice in the index and ensures the owning provider is running and warm.\n- **Lazy vs eager warmup**: providers default to lazy — they enumerate voices\n  in milliseconds, deferring model load until first synth. The daemon spawns\n  them eagerly (`--eager-startup`) for the synth path so the model is loaded\n  before any audio commitment is made to speechd. Run `bin/neural-tts-ctl\n  reload-voices` to rebuild the voice index after dropping reference clips\n  or installing a new provider.\n- **Idle eviction**: the daemon unloads its provider after\n  `supervisor.idle_timeout_seconds` (default 600). Next request re-spawns.\n\n## Prerequisites\n\n- Linux + speech-dispatcher 0.12+\n- KDE Plasma 6 with `qt6-qtspeech-speechd`\n- `pulseaudio-utils` or `pipewire-pulse` (provides `paplay`)\n- `espeak-ng` (Kokoro uses it via the `misaki` G2P library)\n- [`mise`](https://mise.jdx.dev/) to manage tools and run tasks\n\nOn Fedora 44:\n\n```bash\nsudo dnf install speech-dispatcher pulseaudio-utils espeak-ng \\\n                 qt6-qtspeech-speechd\n```\n\n## Quickstart\n\n```bash\n# 1. Create the daemon's venv (numpy, soxr, protobuf)\nmise run sync-daemon\n\n# 2. Install the Kokoro provider (creates providers/kokoro-onnx/.venv,\n#    downloads ~400 MB of model files into ~/.local/share/neural-tts-daemon/models/)\nmise run install-provider kokoro-onnx\n# (NVIDIA GPU only) add CUDA runtime + fp16-gpu model:\n#    mise run install-provider kokoro-onnx --extra gpu\n#    requires the NVIDIA driver + xorg-x11-drv-nvidia-cuda-libs (or system CUDA).\n#    Re-running with --extra gpu after a CPU install upgrades in place and\n#    downloads the GPU-quantised fp16 model that the daemon prefers on CUDA.\n\n# 3. Install the systemd units and speechd module config (no sudo needed)\nmise run install\n\n# 4. Verify\nbin/neural-tts-ctl status\nspd-say -o neural-tts-generic \"Speech dispatcher route working\"\n```\n\nThen in KDE: **System Settings → Accessibility → Text-to-Speech**,\nselect *Speech Dispatcher*, pick a kokoro voice (`af_heart`, `am_adam`, …).\n\n## Tasks\n\n| `mise run …` | Purpose |\n|---|---|\n| `sync-daemon` | Create the daemon venv |\n| `install-provider \u003cname\u003e` | Sync a provider's venv (e.g. `kokoro-onnx`) |\n| `download-models [name]` | Re-fetch model files for a provider |\n| `install` | Install systemd units + speechd module config |\n| `uninstall` | Remove user-scope unit files and module config |\n| `run` | Run daemon in the foreground (binds sockets itself) |\n| `status` | `neural-tts-ctl status` |\n| `reload-voices` | Rebuild the global voice index (re-enumerate every enabled provider) |\n| `voices` | List speechd-visible voices |\n| `logs` | `journalctl --user -u neural-tts.service -f` |\n| `test` | Run pytest |\n| `gen-proto` | Regenerate `*_pb2.py` from `proto/neural_tts.proto` |\n\n## Files installed (user scope)\n\n```\n~/.config/systemd/user/\n    neural-tts.service             ← daemon\n    neural-tts.socket              ← synthesis socket  ($XDG_RUNTIME_DIR/neural-tts.sock)\n    neural-tts-control.socket      ← control socket    ($XDG_RUNTIME_DIR/neural-tts-control.sock)\n~/.config/speech-dispatcher/\n    speechd.conf                ← seeded from /etc/, with AddModule line appended\n    modules/neural-tts-generic.conf\n~/.config/neural-tts-daemon/\n    config.toml                 ← default provider + supervisor settings\n~/.local/share/neural-tts-daemon/\n    models/                     ← Kokoro ONNX model files (~400 MB)\n    voices/                     ← cloned voice reference clips (future)\n```\n\n## Configuration\n\n`~/.config/neural-tts-daemon/config.toml`:\n\n```toml\n[provider]\ndefault = \"kokoro-onnx\"             # vestigial: only used when `eager_startup`\n                                    # below is true; daemon auto-routes by voice id otherwise.\nenabled = [\"kokoro-onnx\"]           # allowlist; providers not listed here are\n                                    # invisible to the daemon. Add other names\n                                    # (e.g. \"moss-tts-nano\") after installing them.\n\n[supervisor]\nidle_timeout_seconds = 600   # unload provider after this many seconds idle (0 = never)\neager_startup = false        # if true, pre-spawn `provider.default` with its model\n                             # loaded at daemon start. Otherwise providers spawn\n                             # on demand when a synth request arrives.\n\n# Optional per-provider settings. Section name matches the provider's\n# registry name (use TOML's bracket form for hyphens, as below).\n[providers.kokoro-onnx]\nvoices = [\"bm_daniel\", \"bm_lewis\"]   # voice-id allowlist; omit or set to []\n                                     # to surface every voice the provider\n                                     # reports. Filtering happens during\n                                     # enumeration, so dropped voices never\n                                     # reach speechd or its clients.\n```\n\nProviders are opt-in: a fresh install ships with only `kokoro-onnx` enabled.\nAfter running `mise run install-provider \u003cname\u003e`, add the name to\n`[provider] enabled` to make its voices visible to the daemon. The daemon\nroutes every synth request to the owning provider via the global voice\nindex — there's no manual \"switch to provider X\" step.\n\nPer-provider voice allowlists (`[providers.\u003cname\u003e] voices = [...]`) let\nyou trim a provider's voice list before it reaches speechd — useful for\nkokoro's 54-voice catalogue when you only want a handful surfaced in\nFirefox/Okular pickers. Use `spd-say -o neural-tts --list-synthesis-voices`\nto see voice ids, then list the ones you want to keep. After editing the\nconfig, restart the daemon (`systemctl --user restart neural-tts.service`)\nto re-enumerate.\n\nPer-provider environment variables (`[providers.\u003cname\u003e] env = { ... }`)\nget injected into the provider's subprocess on spawn. Use this to set\nprovider knobs without editing a systemd drop-in:\n\n```toml\n[providers.omnivoice]\nenv = { TTS_OMNIVOICE_NUM_STEP = \"24\", TTS_OMNIVOICE_DEVICE = \"cuda:0\" }\n```\n\nValues must be strings (TOML's typed values get coerced). The env table\noverrides any same-named variable from the daemon's own environment.\n\nPer-provider locale overrides (`[providers.\u003cname\u003e.locales]`) replace\nthe provider-declared BCP-47 tag on matching voices and can fan a single\nvoice out across multiple locales. Keys are fnmatch globs against the\nvoice id; the first matching glob wins, and the first tag in the list\nbecomes the primary language while the rest are advertised as additional\nlocales in `LIST VOICES`:\n\n```toml\n[providers.kokoro-onnx.locales]\n\"af_*\" = [\"en-US\", \"en-AU\"]\n\"bf_*\" = [\"en-GB\", \"en-AU\", \"en-NZ\"]\n```\n\nEnvironment overrides (set in a service drop-in):\n\n| Var | Default | Effect |\n|---|---|---|\n| `TTS_KOKORO_MODEL_PATH` | auto (fp16-gpu on GPU, int8 on CPU) | Pin a specific Kokoro model file |\n| `TTS_KOKORO_VOICES_PATH` | `~/.local/share/neural-tts-daemon/models/voices-v1.0.bin` | Pin a voices file |\n| `NEURAL_TTS_LOG_LEVEL` | `INFO` | Daemon and provider log level |\n\n## LongCat-AudioDiT (zero-shot voice cloning)\n\n\u003e **Status: untested.** The LongCat provider builds and follows the same\n\u003e daemon/provider protocol as the others, but it has not been end-to-end\n\u003e verified — the 1B model needs more VRAM than the dev box has. Treat the\n\u003e install and synth paths here as unproven; expect rough edges (notably:\n\u003e the model download is not resumable after a network interruption, so\n\u003e reinstall on failure).\n\nLongCat-AudioDiT is a diffusion TTS that clones any voice from a short\nreference clip — no fine-tuning, no per-voice training. It's CUDA-only and\nthe 1B model needs roughly 5–6 GB VRAM at fp16.\n\n```bash\n# 1. Install the provider (creates providers/longcat-audiodit/.venv,\n#    downloads ~4 GB of HuggingFace model files)\nmise run install-provider longcat-audiodit --extra gpu\n\n# 2. Drop reference clips into the user-voices dir. Each \"voice\" is a\n#    pair of files:\n#      \u003cvoice-id\u003e.\u003clang\u003e.wav   ← 5-15 s clean reference audio\n#      \u003cvoice-id\u003e.\u003clang\u003e.txt   ← exact transcript of that clip (UTF-8)\n#    lang must be `en` or `zh`. Optional sidecar:\n#      \u003cvoice-id\u003e.\u003clang\u003e.toml  ← { display_name = \"...\", gender = \"female\" }\nmkdir -p ~/.local/share/neural-tts-daemon/voices/longcat\ncp my-clip.wav ~/.local/share/neural-tts-daemon/voices/longcat/alice.en.wav\necho \"the exact words spoken in my clip\" \\\n    \u003e ~/.local/share/neural-tts-daemon/voices/longcat/alice.en.txt\n\n# 3. Refresh the global voice index (only needed after dropping new clips)\nbin/neural-tts-ctl reload-voices\n\n# 4. Speak — the daemon auto-routes by voice id; no explicit switch needed\nspd-say -o neural-tts -y alice \"Hello world from LongCat.\"\n```\n\nLimitations: English + Chinese only; the daemon's `speed` knob is ignored\n(LongCat has no native speed control); long input is sentence-chunked, so\ntime-to-first-audio scales with the first chunk, not the full text.\n\n## MOSS-TTS-Nano (CPU-friendly zero-shot voice cloning)\n\nMOSS-TTS-Nano is a 100M-parameter autoregressive TTS shipped as ONNX. No\ntorch GPU required at runtime — upstream claims realtime on 4 CPU cores.\nVoice cloning is zero-shot from a single reference clip; no transcript is\nneeded (the model conditions on audio tokens, not text).\n\n```bash\n# 1. Install the provider — clones the pinned upstream into\n#    providers/moss-tts-nano/vendor/, creates its .venv, and downloads\n#    both ONNX repos (~few hundred MB) into ~/.local/share/neural-tts-daemon/models/.\nmise run install-provider moss-tts-nano\n#    The [gpu] extra exists but is NOT recommended for this model:\n#    a 100M-param autoregressive ONNX session on GPU loses to CPU on most\n#    hardware (per-token CUDA launches + memcpy overhead exceed the compute\n#    savings). On one local test, CPU ran 2.4× realtime; GPU ran 1.0×.\n#    To force the choice anyway: TTS_MOSS_TTS_NANO_EP=cpu|cuda (env var).\n\n# 2. Drop reference clips into the user-voices dir. Each \"voice\" is one wav:\n#      \u003cvoice-id\u003e.\u003clang\u003e.wav    ← 5-15 s clean reference audio\n#    lang is the short tag for one of the 20 supported languages:\n#      zh, en, de, es, fr, ja, it, hu, ko, ru, fa, ar, pl, pt, cs, da, sv, el, tr\n#    Optional sidecar:\n#      \u003cvoice-id\u003e.\u003clang\u003e.toml   ← { display_name = \"...\", gender = \"female\" }\nmkdir -p ~/.local/share/neural-tts-daemon/voices/moss-tts-nano\ncp my-clip.wav ~/.local/share/neural-tts-daemon/voices/moss-tts-nano/alice.en.wav\n\n# 3. Refresh the global voice index (only needed after dropping new clips)\nbin/neural-tts-ctl reload-voices\n\n# 4. Speak — the daemon auto-routes by voice id; no explicit switch needed\nspd-say -o neural-tts -y alice \"Hello world from MOSS-TTS-Nano.\"\n```\n\nLimitations: the daemon's `speed` knob is ignored (no native speed control);\ninput is chunked by token budget so time-to-first-audio scales with the\nfirst chunk; WeTextProcessing-based text normalisation is intentionally\ndisabled (it requires `pynini`, which doesn't install cleanly under `uv`),\nso very heavy numeric/abbreviation input may sound less polished than via\nupstream's CLI.\n\n## OmniVoice (massively multilingual zero-shot cloning)\n\nOmniVoice is a diffusion language model TTS from k2-fsa (Apache-2.0). It\nclones voices zero-shot from a short reference clip, supports 600+\nlanguages, and runs on CUDA, Intel Arc (XPU), Apple Silicon (MPS), or CPU\nvia PyTorch. The 1B+ model is too large for realtime CPU synthesis; a GPU\nis strongly recommended.\n\n```bash\n# 1. Install the provider — pulls a pinned commit of upstream from GitHub\n#    via uv, then snapshot-downloads the HuggingFace model weights\n#    (~few GB) into ~/.local/share/neural-tts-daemon/models/omnivoice/.\nmise run install-provider omnivoice --extra gpu\n#    Omit --extra gpu to pull the CPU torch wheel instead (much smaller\n#    download, but synthesis will be well below realtime).\n\n# 2. Drop reference clips into the user-voices dir. Each \"voice\" is one wav:\n#      \u003cvoice-id\u003e.\u003clang\u003e.wav    ← 3-10 s clean reference audio\n#    \u003clang\u003e is a BCP-47 primary subtag (en, fr, zh, sw, …) — OmniVoice\n#    handles 600+ languages, we just pass it through. Optional sidecars:\n#      \u003cvoice-id\u003e.\u003clang\u003e.txt    ← manual transcript (overrides Whisper)\n#      \u003cvoice-id\u003e.\u003clang\u003e.toml   ← { display_name = \"...\", gender = \"female\" }\nmkdir -p ~/.local/share/neural-tts-daemon/voices/omnivoice\ncp my-clip.wav ~/.local/share/neural-tts-daemon/voices/omnivoice/alice.en.wav\n\n# 3. Refresh the global voice index (only needed after dropping new clips)\nbin/neural-tts-ctl reload-voices\n\n# 4. Speak — the daemon auto-routes by voice id; no explicit switch needed\nspd-say -o neural-tts -y alice \"Hello world from OmniVoice.\"\n```\n\nYou can reuse a reference clip from another cloning provider by\nsymlinking it into the omnivoice voices dir, e.g.:\n\n```bash\nln -s ../moss-tts-nano/hobbits.en.wav \\\n      ~/.local/share/neural-tts-daemon/voices/omnivoice/hobbits.en.wav\n```\n\nEnvironment overrides:\n\n| Var | Default | Effect |\n|---|---|---|\n| `TTS_OMNIVOICE_DEVICE` | auto (cuda → xpu → mps → cpu) | Pin a specific torch device string |\n| `TTS_OMNIVOICE_MODEL_PATH` | `~/.local/share/neural-tts-daemon/models/omnivoice` | Pin a local model snapshot dir |\n| `TTS_OMNIVOICE_NUM_STEP` | `16` | Diffusion steps per utterance. Quality vs latency knob — upstream default is 32 (better fidelity); 16 is the README's faster-inference value. Lower = lower TTFA + per-chunk synth time; higher = cleaner audio. |\n| `TTS_OMNIVOICE_COMPILE` | `` (off) | Wrap the model in `torch.compile()` for ~20-40% faster steady-state per diffusion step. Costs 30-60 s extra on the first synth (JIT compile). Accepts `1`/`true`/`on` (= `default` mode), or one of `default`, `reduce-overhead`, `max-autotune`. `reduce-overhead` is fastest but uses CUDA graphs that pin tensor shapes — if you hit recompile storms or shape errors, drop to `default`. Compile failures fall back to eager with a warning. |\n\nLimitations: no native streaming API upstream, so input is sentence-chunked\nand emitted per chunk (time-to-first-audio scales with the first chunk).\nOn a fresh voice without a transcript sidecar, the first synthesis pays a\none-time Whisper transcription cost; subsequent calls reuse it from\nin-memory cache.\n\n## Qwen3-TTS (streaming zero-shot cloning, CUDA-only)\n\nQwen3-TTS-0.6B via [faster-qwen3-tts](https://github.com/andimarafioti/faster-qwen3-tts) —\na hand-written CUDA-graph capture of Qwen3-TTS's predictor + talker that\nhits ~4.8× realtime on an RTX 4090 (with ~150 ms TTFA) and well into\nmulti-x on smaller cards too. No flash-attn, no vLLM, no Triton; just\n`torch.cuda.CUDAGraph` over a static KV cache. Streams natively, so by\ndefault the provider passes each model chunk straight to the daemon.\n\nVoice cloning requires both a reference WAV and a transcript sidecar (no\ntranscript-less mode in v1). 10 languages.\n\n```bash\n# 1. Install the provider — pulls faster-qwen3-tts from PyPI and\n#    snapshot-downloads the HF model (~1.5 GB) into\n#    ~/.local/share/neural-tts-daemon/models/qwen3-tts-0.6b/.\nmise run install-provider qwen3-tts --extra gpu\n\n# 2. Drop reference clips into the user-voices dir. Each \"voice\" is two\n#    files (transcript is REQUIRED):\n#      \u003cvoice-id\u003e.\u003clang\u003e.wav    ← 3-10 s clean reference audio\n#      \u003cvoice-id\u003e.\u003clang\u003e.txt    ← exact transcript of the wav\n#    \u003clang\u003e must be one of: en, zh, ja, ko, de, fr, ru, pt, es, it.\n#    Optional sidecar:\n#      \u003cvoice-id\u003e.\u003clang\u003e.toml   ← { display_name = \"...\", gender = \"female\" }\nmkdir -p ~/.local/share/neural-tts-daemon/voices/qwen3-tts\ncp my-clip.wav ~/.local/share/neural-tts-daemon/voices/qwen3-tts/alice.en.wav\necho \"exact transcript of my-clip\" \\\n  \u003e ~/.local/share/neural-tts-daemon/voices/qwen3-tts/alice.en.txt\n\n# 3. Refresh the global voice index (only needed after dropping new clips)\nbin/neural-tts-ctl reload-voices\n\n# 4. Speak\nspd-say -o neural-tts -y alice \"Hello world from Qwen3-TTS.\"\n```\n\nEnvironment overrides:\n\n| Var | Default | Effect |\n|---|---|---|\n| `TTS_QWEN3_DEVICE` | auto (`cuda` if available) | Pin device, e.g. `cuda:1`. CPU is unsupported by faster-qwen3-tts. |\n| `TTS_QWEN3_MODEL_PATH` | `~/.local/share/neural-tts-daemon/models/qwen3-tts-0.6b` | Pin a local model snapshot dir |\n| `TTS_QWEN3_DTYPE` | `bf16` | One of `bf16`, `fp16`, `fp32`. bf16 is the upstream sweet spot. |\n| `TTS_QWEN3_ATTN` | `sdpa` | `sdpa` or `flash_attention_2`. Flash-attn isn't installed by default; sdpa is fine. |\n| `TTS_QWEN3_MAX_SEQ_LEN` | `2048` | KV-cache capacity. Bigger = more VRAM. |\n| `TTS_QWEN3_GREEDY` | `` (off) | `1` switches from sampled to greedy decoding: always pick the highest-probability next token. Modest speedup, fully deterministic output, slightly less prosodic variation. Usually a win for voice-clone TTS because the reference clip pins the voice character; drop it if synthesis sounds robotic. |\n| `TTS_QWEN3_TEMPERATURE` | `0.9` | Sampled-decoding temperature. Ignored when `GREEDY=1`. |\n| `TTS_QWEN3_TOP_K` | `50` | Sampled-decoding top-k. Ignored when `GREEDY=1`. |\n| `TTS_QWEN3_TOP_P` | `1.0` | Sampled-decoding top-p. Ignored when `GREEDY=1`. |\n| `TTS_QWEN3_REPETITION_PENALTY` | `1.05` | Repetition penalty applied to the predictor. |\n| `TTS_QWEN3_CHUNK_SIZE` | `12` | Codec-frame batching per yield from the streaming generator. Lower = lower TTFA, slightly more overhead per yield. |\n| `TTS_QWEN3_MAX_NEW_TOKENS_PER_CHAR` | `6` | Cap on `max_new_tokens` is `min(2048, 32 + per_char * len(text))`. Lower = tighter cap (faster) but risks truncating long sentences. |\n\n**Optional RTF-aware jitter buffer.** On sub-realtime hardware (sustained\nRTF \u003c 1, e.g. an RTX 3060 Laptop on the 0.6B model) raw streaming starves\n`paplay` mid-utterance and you get audible gaps. Setting\n`TTS_QWEN3_CHUNKER=1` enables a sentence-aware chunker plus an adaptive\nprebuffer per chunk: it measures RTF from the model's per-yield timing\ndict, estimates how much audio to hold back before emitting, then drains\nthe buffer and passes through the rest of the chunk. Trades TTFA for\ngap-free playback; off by default because faster GPUs don't need it.\n\n| Var | Default | Effect |\n|---|---|---|\n| `TTS_QWEN3_CHUNKER` | `` (off) | `1` enables the chunker + jitter buffer described above. |\n| `TTS_QWEN3_CHUNK_TARGET_CHARS` | `120` | Target characters per chunk. Smaller = lower TTFA + more prosody breaks. |\n| `TTS_QWEN3_CHUNK_HARD_CAP_CHARS` | `240` | Hard ceiling per chunk after soft-break fallback. |\n| `TTS_QWEN3_JITTER_SAFETY_MS` | `200` | Extra millis added on top of the computed prebuffer to absorb GPU contention spikes. |\n| `TTS_QWEN3_JITTER_INITIAL_MS` | `500` | Prebuffer used for the very first chunk before any RTF has been observed. |\n| `TTS_QWEN3_CHARS_PER_SEC_BOOTSTRAP` | `15.0` | Initial chars/sec estimate before the first chunk's audio duration calibrates the ratio. |\n\nMath: for a chunk of audio duration D at sustained RTF r \u003c 1, gap-free\nplayback requires a prebuffer ≥ D·(1−r)/r. At r = 0.5 that's D — i.e.\nhold the whole chunk before emitting. Streaming-within-chunk still wins\nover full pre-synth because the model keeps producing during playback;\ntotal wall time per chunk is D/r vs D/r + D for full pre-synth, so TTFA\nhalves.\n\nLimitations: CUDA-required; transcript sidecar mandatory; no\n`speed`/style knob; 10 languages only (see `LANG_TO_QWEN` in\n`voices.py`); first synth pays a one-time CUDA-graph capture cost\n(~10–30 s).\n\n## Adding another provider\n\nA provider is a uv project under `providers/\u003cname\u003e/` with:\n\n1. `pyproject.toml` declaring its deps (torch, ONNX runtime, whatever).\n2. A Python module that:\n   - Reads `NEURAL_TTS_PROVIDER_FD` env var, adopts that FD as an `AF_UNIX` socket.\n   - Runs a framed-protobuf request/response loop (`proto/neural_tts.proto`).\n   - Implements `Warmup`, `Synthesize`, `ListVoices`, `Shutdown`.\n3. An entry in `providers/registry.toml` mapping the provider name to its\n   project dir and Python module name.\n\nAdd a future cloning-capable provider's `register_cloned_voice` and\n`remove_cloned_voice` ops once the upstream lib is ready.\n\n## Troubleshooting\n\n- **`bin/neural-tts-ctl status` says state=STOPPED for ages**: that's normal —\n  the provider only spawns on the first synthesis request (or set\n  `eager_startup = true` in config.toml).\n- **KDE TTS settings shows no Kokoro voices**: after install you may need to\n  restart any open KDE apps so they re-query speech-dispatcher. The install\n  script kills running `speech-dispatcher` processes so the next client\n  respawn re-reads the new config — but apps with long-lived speechd\n  connections (open Okular, etc.) need to be restarted.\n- **First utterance after a long pause is slow**: that's the idle-evict\n  reload (default 10 min). Increase `idle_timeout_seconds` or set to `0` if\n  you have RAM to spare.\n- **`uv sync` is slow the first time**: kokoro-onnx pulls onnxruntime\n  (~80 MB wheel) and misaki/espeak-loader. Subsequent syncs hit the uv cache.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkellpossible%2Fneural-tts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkellpossible%2Fneural-tts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkellpossible%2Fneural-tts/lists"}