{"id":50418277,"url":"https://github.com/lahirumaramba/qwen-tts-cloner","last_synced_at":"2026-05-31T07:02:38.322Z","repository":{"id":358559464,"uuid":"1241870153","full_name":"lahirumaramba/qwen-tts-cloner","owner":"lahirumaramba","description":"Qwen-TTS Cloner (Google Colab Edition), Demo: https://youtu.be/tMpZ1kpeqoA","archived":false,"fork":false,"pushed_at":"2026-05-17T23:14:56.000Z","size":1891,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-18T01:30:36.042Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://youtu.be/tMpZ1kpeqoA","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lahirumaramba.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-17T23:07:57.000Z","updated_at":"2026-05-17T23:14:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/lahirumaramba/qwen-tts-cloner","commit_stats":null,"previous_names":["lahirumaramba/qwen-tts-cloner"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/lahirumaramba/qwen-tts-cloner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lahirumaramba%2Fqwen-tts-cloner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lahirumaramba%2Fqwen-tts-cloner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lahirumaramba%2Fqwen-tts-cloner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lahirumaramba%2Fqwen-tts-cloner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lahirumaramba","download_url":"https://codeload.github.com/lahirumaramba/qwen-tts-cloner/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lahirumaramba%2Fqwen-tts-cloner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33722156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-31T07:02:38.254Z","updated_at":"2026-05-31T07:02:38.315Z","avatar_url":"https://github.com/lahirumaramba.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎙️ Qwen-TTS Cloner (Google Colab Edition)\n\n[![Google Colab](https://img.shields.io/badge/Google%20Colab-Active-orange.svg?logo=googlecolab)](https://colab.research.google.com/)\n[![Google Drive](https://img.shields.io/badge/Google%20Drive-Integrated-blue.svg?logo=googledrive)](https://drive.google.com/)\n[![TTS Engine](https://img.shields.io/badge/Piper--TTS-16kHz%20Offline-green.svg)](https://github.com/rhasspy/piper)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Labs](https://img.shields.io/badge/labs-experimental-red.svg)](./labs/)\n\nA voice cloning pipeline that leverages **Qwen3-TTS** model family and the **Piper TTS** training engine. \n\nThis repository is optimized for **Google Colab**, offering a workflow to generate large custom voice datasets and train a lightweight, offline-capable **Piper TTS** model. Direct local execution has been quarantined into the `labs/` folder due to the high VRAM requirements and complex dependencies that struggle to run on my MacBook :).\n\n---\n\n## 🗺️ System Workflow\n\n```mermaid\ngraph TD\n    %% Setup Drive\n    SubGraph1[1. Google Drive Input Setup]\n    A[reference.wav \u003cbr\u003e transcript.txt \u003cbr\u003e sentences.txt] --\u003e|Upload to Drive| B[MyDrive/tts_input/]\n\n    %% Notebook 1\n    SubGraph2[2. Dataset Generation]\n    B --\u003e|Mount Drive| C(qwen_tts_dataset_generator.ipynb)\n    C --\u003e|Qwen3-TTS Inference| D[MyDrive/piper_dataset/ \u003cbr\u003e wavs/ \u0026 metadata.csv]\n\n    %% Notebook 2\n    SubGraph3[3. Model Fine-Tuning]\n    D --\u003e|Local VM copy| E(piper_train_colab.ipynb)\n    E --\u003e|Train on Fast Local NVMe VM Disk| F[Colab Storage: /content/piper_logs]\n    F --\u003e|Smart Non-Blocking Backup| G[MyDrive/piper_training_logs]\n    F --\u003e|Compile \u0026 Export ONNX| H[MyDrive/rocky_model_final.onnx \u003cbr\u003e rocky_model_final.onnx.json]\n\n    %% Local Deployment\n    SubGraph4[4. Offline Local Deployment]\n    H --\u003e|Download| I[Offline Low-Power Devices \u003cbr\u003e e.g. Raspberry Pi, Mac, PC]\n    I --\u003e|Ultra-Low Latency Inference| J[Local Synthesized Speech]\n\n    style B fill:#d1ecf1,stroke:#17a2b8,stroke-width:1px\n    style D fill:#d1ecf1,stroke:#17a2b8,stroke-width:1px\n    style H fill:#d4edda,stroke:#28a745,stroke-width:2px\n    style J fill:#f8d7da,stroke:#dc3545,stroke-width:1px\n```\n\n---\n\n## 🚀 Google Colab Quick-Start Guide\n\nTo train your custom voice in the cloud using Google Colab and then run the resulting voice model completely offline on any lightweight local computer, follow these four steps:\n\n### Step 1: Prepare and Upload Inputs to Google Drive\nCreate a folder named `tts_input` in your Google Drive root directory. \nUpload the following three files (examples of which are provided in this repository):\n\n1. **`reference.wav`**: A clean, 3-to-10-second audio recording of the target speaker.\n2. **`transcript.txt`**: A text file containing the exact words spoken in `reference.wav`.\n3. **`sentences.txt`**: A text file containing a list of sentences (recommended: 100 to 500 prompts) you want the cloned voice to say to create the training corpus.\n\n\u003e [!TIP]\n\u003e **Voice Quality Guidelines**\n\u003e For perfect voice clones, ensure `reference.wav` is completely \"dry\" (no background music, no echoes, no static noise) and recorded with a clear, standard microphone.\n\n---\n\n### Step 2: Batch Dataset Generation\nOpen [qwen_tts_dataset_generator.ipynb](qwen_tts_dataset_generator.ipynb) in Google Colab (tested on **A100 GPU 2025.10 High-RAM** runtime):\n\n1. **Mount Drive**: The notebook will guide you to authorize Google Drive access.\n2. **Configure Paths**: Point to your uploaded inputs:\n   * Reference Audio: `/content/drive/MyDrive/tts_input/reference.wav`\n   * Transcript File: `/content/drive/MyDrive/tts_input/transcript.txt`\n   * Targets List: `/content/drive/MyDrive/tts_input/sentences.txt`\n3. **Run Batch Synthesis**: The cell downloads the lightweight `qwen-tts` framework and synthesizes all lines in `sentences.txt` in the cloned voice.\n4. **Result**: Your Google Drive is populated with an LJSpeech-formatted dataset inside `/MyDrive/piper_dataset` (containing `wavs/` and a normalized `metadata.csv`).\n\n---\n\n### Step 3: Standalone Piper Model Fine-Tuning\nOpen [piper_train_colab.ipynb](piper_train_colab.ipynb) in Google Colab (tested on **A100 GPU 2025.10 High-RAM** runtime):\n\n\u003e [!IMPORTANT]\n\u003e **Google Drive API Protection**\n\u003e Training checkpoints and PyTorch Lightning logging write highly frequent I/O operations. Writing these directly to mounted Google Drive folders triggers severe API rate limits and locks up training.\n\u003e\n\u003e **The Fix**: This notebook performs all training on the **Colab Local VM Storage** (`/content/piper_logs`). A specialized script then performs non-blocking smart backups of your training progress back to Google Drive (`/MyDrive/piper_training_logs`) allowing safe resumes across sessions.\n\n1. **Preprocessing**: The notebook automatically resamples the Qwen-generated audio from 24kHz down to 16kHz (optimized for Piper low/medium quality tiers) and formats it.\n2. **Base Checkpoint**: Downloads the pre-trained `Lessac Low` checkpoint to begin transfer learning.\n3. **Training Execution**: Fine-tunes the model. A Tensorboard widget displays live training loss.\n4. **ONNX Compilation**: The final cell compiles the absolute latest checkpoint (`.ckpt`) into an optimized `.onnx` graph and packs it with its matching `.onnx.json` configuration file, dropping both in your Google Drive root.\n\n---\n\n### Step 4: Local Offline Deployment\nOnce training is finished, download the compiled `.onnx` and `.onnx.json` files from your Google Drive. You can run high-fidelity text-to-speech completely offline locally on Raspberry Pi!\n\n```bash\necho \"Science saves the world. It's a good plan.\" | \\\n  piper --model my_custom_model.onnx \\\n        --output_file custom_voice_output.wav\n```\n\n**Piper TTS Demo Video:**\u003cbr\u003e\n\u003ca href=\"https://youtu.be/tMpZ1kpeqoA\" target=\"_blank\"\u003e\n  \u003cimg src=\"thumbnail_tts.png\" alt=\"Piper TTS Demo on YouTube\" width=\"600\"\u003e\n\u003c/a\u003e\n\n---\n\n## 📂 Project Structure\n\n```text\n├── qwen_tts_dataset_generator.ipynb  # Primary: Generates LJSpeech datasets via Colab\n├── piper_train_colab.ipynb           # Primary: Fine-tunes and exports Piper models on Colab\n├── sentences.txt                     # calibration sentences (500 lines) for dataset generation\n├── transcript.txt                    # Reference transcript template\n├── labs/                             # QUARANTINED experimental local execution tools\n│   ├── clone_voice.py                # Local zero-shot voice cloning CLI tool\n│   ├── pyproject.toml                # UV environment definitions\n│   ├── uv.lock                       # Lockfile for direct local environment sync\n│   └── .python-version               # Python version target (3.14)\n└── README.md                         # This file\n```\n\n---\n\n## 🧪 Labs: Experimental Local Voice Cloning\n\nIf you have a powerful local system, you can run local synthesis in the `labs/` directory.\n\n\u003e [!WARNING]\n\u003e Local cloning is **highly experimental** and prone to Out-Of-Memory (OOM) errors, complex dependency resolution, and slower execution on standard laptops. I haven't successfully tested this on a local system yet.\n\n### Run Local CLI Cloning\nEnsure you have [uv](https://docs.astral.sh/uv/) installed:\n```bash\n# Navigate to labs\ncd labs\n\n# Synchronize virtual env\nuv sync\n\n# Run cloning CLI\nuv run clone_voice.py \\\n    --ref_audio \"../reference.wav\" \\\n    --ref_text \"Exactly what is said in reference audio.\" \\\n    --gen_text \"Synthesizing this text using local local resources.\" \\\n    --output \"local_output.wav\"\n```\n\n---\n\n## 📜 Credits\n\n* **Qwen3-TTS**: The model architecture is built by [Qwen Team](https://github.com/QwenLM/Qwen3-TTS).\n* **Piper TTS**: Fast, local neural text-to-speech system developed by [Rhasspy](https://github.com/rhasspy/piper).\n\n---\n\n## ⚖️ License\n\nThis project is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flahirumaramba%2Fqwen-tts-cloner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flahirumaramba%2Fqwen-tts-cloner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flahirumaramba%2Fqwen-tts-cloner/lists"}