{"id":45187855,"url":"https://github.com/second-state/qwen3_tts_rs","last_synced_at":"2026-04-02T17:43:20.318Z","repository":{"id":334679107,"uuid":"1141503458","full_name":"second-state/qwen3_tts_rs","owner":"second-state","description":"A Rust implementation of the Qwen3 Text-to-Speech (TTS) model inference.","archived":false,"fork":false,"pushed_at":"2026-03-28T08:31:44.000Z","size":3440,"stargazers_count":194,"open_issues_count":2,"forks_count":28,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-28T12:25:44.871Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/second-state.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-25T00:00:55.000Z","updated_at":"2026-03-28T08:31:35.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/second-state/qwen3_tts_rs","commit_stats":null,"previous_names":["juntao/qwen3_tts_rs","second-state/qwen3_tts_rs"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/second-state/qwen3_tts_rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/second-state%2Fqwen3_tts_rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/second-state%2Fqwen3_tts_rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/second-state%2Fqwen3_tts_rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/second-state%2Fqwen3_tts_rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/second-state","download_url":"https://codeload.github.com/second-state/qwen3_tts_rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/second-state%2Fqwen3_tts_rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31312361,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-20T11:05:03.208Z","updated_at":"2026-04-02T17:43:20.311Z","avatar_url":"https://github.com/second-state.png","language":"Rust","funding_links":[],"categories":["Voice AI"],"sub_categories":[],"readme":"# Qwen3 TTS - Rust CLI tools\n\n[![Crates.io](https://img.shields.io/crates/v/qwen3-tts-rs.svg)](https://crates.io/crates/qwen3-tts-rs)\n[![License](https://img.shields.io/crates/l/qwen3-tts-rs.svg)](https://github.com/second-state/qwen3_tts_rs/blob/main/LICENSE)\n\nA Rust implementation of the Qwen3 Text-to-Speech (TTS) model inference. Provides three cross-platform CLI tools suitable for agentic skills for AI agents and bots.\n\n- **tts** — generate speech from text with named speaker voices\n- **voice_clone** — clone a voice from reference audio\n- **api_server** — OpenAI-compatible HTTP API server\n\nSupports two backends: **libtorch** (via the `tch` crate, cross-platform with optional CUDA) and **MLX** (Apple Silicon native via Metal GPU).\n\nLearn more:\n* [A Rust implementation / CLI](https://github.com/second-state/qwen3_asr_rs) for Qwen3's ASR (Automatic Speech Recognition or Speech-to-Text) models\n* An OpenAI compatible [API server for audio / speech](https://github.com/second-state/qwen3_audio_api/tree/main/rust)\n* An OpenClaw SKILL for voice generation. Copy and Paste to your lobster to [install it](https://raw.githubusercontent.com/second-state/qwen3_tts_rs/refs/heads/main/skills/install.md)\n\n## Quick Start\n\nInstall binaries, models, and reference audio for your platform:\n\n```bash\ncurl -sSf https://raw.githubusercontent.com/second-state/qwen3_tts_rs/main/install.sh | bash\ncd qwen3_tts_rs\n```\n\nThe installer detects your OS, CPU, and NVIDIA GPU (if present), then sets up everything in `./qwen3_tts_rs/`.\n\n### Text-to-Speech\n\nGenerate speech with a named speaker using the CustomVoice model:\n\n```bash\n./tts models/Qwen3-TTS-12Hz-0.6B-CustomVoice \"Hello world, this is a test.\" Vivian english\n# Output: output.wav (24 kHz)\n```\n\n### Voice Cloning\n\nClone a voice from reference audio using the Base model (ICL mode):\n\n```bash\n./voice_clone models/Qwen3-TTS-12Hz-0.6B-Base reference_audio/trump.wav \\\n  \"Hello, this is a voice cloning test.\" english \\\n  \"Angered and appalled millions of Americans across the political spectrum\"\n# Output: output_voice_clone.wav (24 kHz)\n```\n\n### API Server\n\nStart the OpenAI-compatible API server with the CustomVoice model:\n\n```bash\n./api_server models/Qwen3-TTS-12Hz-0.6B-CustomVoice --port 8080\n```\n\nThen call the endpoint:\n\n```bash\ncurl -X POST http://localhost:8080/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"input\": \"Hello world!\", \"voice\": \"alloy\"}' \\\n  -o output.wav\n```\n\n## Reference\n\n### `tts` — Text-to-Speech\n\n```\ntts \u003cmodel_path\u003e [text] [speaker] [language] [instruction]\n```\n\n| Argument | Default | Description |\n|----------|---------|-------------|\n| `model_path` | (required) | Path to model directory |\n| `text` | \"Hello! This is a test...\" | Text to synthesize (max ~4096 chars) |\n| `speaker` | Vivian | Speaker name (see below) |\n| `language` | english | Language: `english`, `chinese`, `japanese`, `korean` |\n| `instruction` | (empty) | Voice style instruction (1.7B models only) |\n\nOutput: `output.wav` (24 kHz, 16-bit PCM)\n\n**Available speakers** (CustomVoice models): Vivian, Serena, Ryan, Aiden, Uncle_fu, Ono_anna, Sohee, Eric, Dylan\n\n**Instruction examples** (1.7B CustomVoice only):\n- `\"Speak in an urgent and excited voice\"`\n- `\"Speak happily and joyfully\"`\n- `\"Speak slowly and calmly\"`\n- `\"Speak in a whisper\"`\n\n### `voice_clone` — Voice Cloning\n\n```\nvoice_clone \u003cmodel_path\u003e \u003cref_audio\u003e [text] [language] [ref_text]\n```\n\n| Argument | Default | Description |\n|----------|---------|-------------|\n| `model_path` | (required) | Path to model directory |\n| `ref_audio` | (required) | Path to reference WAV file |\n| `text` | \"Hello! This is a test...\" | Text to synthesize |\n| `language` | english | Language |\n| `ref_text` | (none) | Transcript of reference audio (enables ICL mode, higher quality) |\n\nOutput: `output_voice_clone.wav` (24 kHz, 16-bit PCM)\n\n**Preparing reference audio:** Must be mono 24 kHz 16-bit WAV. Convert with ffmpeg:\n\n```bash\nffmpeg -i input.m4a -ac 1 -ar 24000 -sample_fmt s16 reference.wav\n```\n\n**Example:**\n\n```bash\n./voice_clone models/Qwen3-TTS-12Hz-0.6B-Base reference_audio/trump.wav \\\n  \"Hello, this is a voice cloning test.\" english \\\n  \"Angered and appalled millions of Americans across the political spectrum\"\n```\n\n### `api_server` — OpenAI-Compatible API\n\n```\napi_server \u003cmodel_path\u003e [--host 127.0.0.1] [--port 8080]\n```\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `model_path` | (required) | Path to model directory |\n| `--host` | 127.0.0.1 | Bind address |\n| `--port` | 8080 | Listen port |\n\n**Endpoints:**\n\n| Method | Path | Description |\n|--------|------|-------------|\n| POST | `/v1/audio/speech` | Generate speech (OpenAI-compatible) |\n| GET | `/v1/models` | List available models |\n| GET | `/health` | Health check |\n\n### API Request: `POST /v1/audio/speech`\n\n```json\n{\n  \"input\": \"Text to synthesize\",\n  \"voice\": \"alloy\",\n  \"model\": \"qwen3-tts\",\n  \"response_format\": \"wav\",\n  \"speed\": 1.0,\n  \"stream\": false,\n  \"language\": \"english\",\n  \"instructions\": \"Speak urgently\",\n  \"audio_sample\": \"\u003cbase64-encoded WAV\u003e\",\n  \"audio_sample_text\": \"Transcript of the reference audio\"\n}\n```\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `input` | string | (required) | Text to synthesize (max 4096 chars) |\n| `voice` | string | `\"alloy\"` | OpenAI name or Qwen3 speaker name (see mapping below) |\n| `model` | string | — | Accepted for compatibility, ignored |\n| `response_format` | string | `\"wav\"` | `\"wav\"`, `\"pcm\"`, `\"mp3\"`, `\"flac\"`, `\"ogg\"`, or `\"opus\"` |\n| `speed` | float | 1.0 | Speed multiplier (0.25–4.0) |\n| `stream` | bool | false | Enable SSE streaming (requires `\"pcm\"`) |\n| `language` | string | `\"english\"` | `english`, `chinese`, `japanese`, `korean`, `auto` |\n| `instructions` | string | — | Voice style instruction (1.7B models only) |\n| `audio_sample` | string | — | Base64-encoded reference WAV for voice cloning |\n| `audio_sample_text` | string | — | Transcript of reference audio (required with `audio_sample`) |\n\n**Voice name mapping** (OpenAI → Qwen3):\n\n| OpenAI | Qwen3 |\n|--------|-------|\n| alloy | serena |\n| echo | ryan |\n| fable | vivian |\n| onyx | eric |\n| nova | ono_anna |\n| shimmer | sohee |\n\nYou can also pass Qwen3 speaker names directly (e.g., `\"voice\": \"vivian\"`).\n\n**Streaming:** When `stream: true` and `response_format: \"pcm\"`, the server returns Server-Sent Events with base64-encoded PCM chunks:\n\n```\ndata: {\"type\":\"speech.audio.delta\",\"delta\":\"\u003cbase64 PCM\u003e\"}\ndata: {\"type\":\"speech.audio.done\"}\n```\n\n**Voice cloning via API:**\n\n```bash\n# Encode reference audio as base64\nREF_B64=$(base64 \u003c reference.wav)\n\ncurl -X POST http://localhost:8080/v1/audio/speech \\\n  -H \"Content-Type: application/json\" \\\n  -d \"{\n    \\\"input\\\": \\\"Hello from a cloned voice.\\\",\n    \\\"voice\\\": \\\"alloy\\\",\n    \\\"audio_sample\\\": \\\"$REF_B64\\\",\n    \\\"audio_sample_text\\\": \\\"Transcript of the reference audio\\\"\n  }\" -o cloned.wav\n```\n\n## Build from Source\n\n### macOS (MLX backend)\n\nRequires Apple Silicon Mac, Xcode, and CMake.\n\n```bash\nbrew install cmake\ngit clone https://github.com/second-state/qwen3_tts_rs.git\ncd qwen3_tts_rs\ngit submodule update --init --recursive\ncargo build --release --no-default-features --features mlx\n```\n\n### Linux (libtorch backend)\n\n**1. Download libtorch** from [libtorch-releases](https://github.com/second-state/libtorch-releases/releases/tag/v2.7.1):\n\n```bash\n# Linux x86_64 (CPU)\ncurl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-x86_64-2.7.1.tar.gz\ntar xzf libtorch-cxx11-abi-x86_64-2.7.1.tar.gz\n\n# Linux x86_64 (CUDA 12.6)\ncurl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-x86_64-cuda12.6-2.7.1.tar.gz\ntar xzf libtorch-cxx11-abi-x86_64-cuda12.6-2.7.1.tar.gz\n\n# Linux ARM64 (CPU)\ncurl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-2.7.1.tar.gz\ntar xzf libtorch-cxx11-abi-aarch64-2.7.1.tar.gz\n\n# Linux ARM64 (CUDA 12.6 / Jetson)\ncurl -LO https://github.com/second-state/libtorch-releases/releases/download/v2.7.1/libtorch-cxx11-abi-aarch64-cuda12.6-2.7.1.tar.gz\ntar xzf libtorch-cxx11-abi-aarch64-cuda12.6-2.7.1.tar.gz\n```\n\n**2. Set environment and build:**\n\n```bash\nexport LIBTORCH=$(pwd)/libtorch\nexport LIBTORCH_BYPASS_VERSION_CHECK=1\n\ngit clone https://github.com/second-state/qwen3_tts_rs.git\ncd qwen3_tts_rs\ncargo build --release\n```\n\nAlternatively, use pip-installed PyTorch instead of downloading libtorch:\n\n```bash\npip install torch==2.7.1\nexport LIBTORCH_USE_PYTORCH=1\nexport LD_LIBRARY_PATH=$(python3 -c \"import torch; print(torch.__path__[0])\")/lib:$LD_LIBRARY_PATH\n```\n\n### Download models and generate tokenizer\n\nAfter building, download models and generate `tokenizer.json` for each:\n\n```bash\npip install huggingface_hub transformers\n\nhuggingface-cli download Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice --local-dir models/Qwen3-TTS-12Hz-0.6B-CustomVoice\n\npython3 -c \"\nfrom transformers import AutoTokenizer\nfor model in ['Qwen3-TTS-12Hz-0.6B-CustomVoice', 'Qwen3-TTS-12Hz-0.6B-Base', 'Qwen3-TTS-12Hz-1.7B-CustomVoice']:\n    path = f'models/{model}'\n    try:\n        tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)\n        tok.backend_tokenizer.save(f'{path}/tokenizer.json')\n        print(f'Saved {path}/tokenizer.json')\n    except Exception as e:\n        print(f'Skipped {model}: {e}')\n\"\n```\n\nBinaries are in `target/release/`: `tts`, `voice_clone`, `api_server`.\n\n### Rust library usage\n\nAdd to your `Cargo.toml`:\n\n```toml\n[dependencies]\nqwen3-tts-rs = \"0.2\"\n# Or for MLX backend:\n# qwen3-tts-rs = { version = \"0.2\", default-features = false, features = [\"mlx\"] }\n```\n\nSee the [API documentation on docs.rs](https://docs.rs/qwen3-tts-rs) for library usage examples.\n\n## Performance (Apple M4 Mac, MLX backend)\n\nTest sentences: ~15–20 words in English (\"The quick brown fox...\" / \"Scientists have discovered...\") and Chinese.\n**RTF** = Real-Time Factor (wall time / audio duration). Lower is better; \u003c 1.0 means faster than real-time.\n\n### 0.6B CustomVoice\n\n#### CLI (`tts` / `voice_clone`)\n\n| Test | Speaker | Language | Audio | Wall Time | RTF |\n|------|---------|----------|-------|-----------|-----|\n| Preset voice | Vivian | English | 5.92s | 10.93s | 1.85x |\n| Preset voice | Ryan | English | 8.16s | 14.15s | 1.73x |\n| Preset voice | Vivian | Chinese | 6.64s | 11.26s | 1.70x |\n| Voice clone (ICL) | ref audio | English | 7.04s | 16.77s | 2.38x |\n\n#### API server (after warmup)\n\n| Test | Voice | Mode | Audio | Wall Time | RTF |\n|------|-------|------|-------|-----------|-----|\n| Non-streaming WAV | alloy (serena) | full | 6.40s | 9.90s | 1.55x |\n| Non-streaming WAV | echo (ryan) | full | 8.00s | 12.70s | 1.59x |\n| Streaming PCM | alloy (serena) | stream | ~6.4s | 10.22s | ~1.60x |\n| Voice clone WAV | alloy + ref | full | 9.04s | 20.11s | 2.22x |\n\n### 1.7B CustomVoice\n\n#### CLI (`tts`)\n\n| Test | Speaker | Language | Audio | Wall Time | RTF |\n|------|---------|----------|-------|-----------|-----|\n| Preset voice | Vivian | English | 6.24s | 18.92s | 3.03x |\n| Preset voice | Ryan | English | 8.64s | 27.50s | 3.18x |\n| Preset voice | Vivian | Chinese | 6.24s | 18.31s | 2.93x |\n| Preset + instruction | Vivian | English | 8.80s | 29.97s | 3.41x |\n\n#### API server (after warmup)\n\n| Test | Voice | Mode | Audio | Wall Time | RTF |\n|------|-------|------|-------|-----------|-----|\n| Non-streaming WAV | alloy (serena) | full | 5.52s | 15.14s | 2.74x |\n| Non-streaming WAV | echo (ryan) | full | 8.64s | 26.31s | 3.05x |\n| Streaming PCM | alloy (serena) | stream | 8.16s | 19.79s | 2.43x |\n\n### 0.6B vs 1.7B comparison\n\n| Metric | 0.6B avg RTF | 1.7B avg RTF | Slowdown |\n|--------|-------------|-------------|----------|\n| CLI preset voice | 1.76x | 3.05x | ~1.7x |\n| API non-streaming | 1.57x | 2.90x | ~1.8x |\n\n## Architecture\n\n```\nText → Tokenizer → Dual-stream Embeddings → TalkerModel (28-layer Transformer)\n                                                    ↓\n                                              codec_head → Code 0\n                                                    ↓\n                                        CodePredictor (5-layer Transformer) → Codes 1-15\n                                                    ↓\n                                              Vocoder → 24kHz Waveform\n```\n\n## License\n\nApache-2.0\n\n## Credits\n\nBased on the original Python implementation by the Alibaba Qwen team.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsecond-state%2Fqwen3_tts_rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsecond-state%2Fqwen3_tts_rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsecond-state%2Fqwen3_tts_rs/lists"}