{"id":13825235,"url":"https://github.com/bnsantoso/sub-to-audio","last_synced_at":"2025-07-08T21:31:37.891Z","repository":{"id":181874147,"uuid":"667347940","full_name":"bnsantoso/sub-to-audio","owner":"bnsantoso","description":"Subtitle to audio, generate audio from any subtitle file using Coqui-ai TTS and synchronize the audio timing according to subtitle time.","archived":false,"fork":false,"pushed_at":"2023-12-14T19:26:56.000Z","size":102,"stargazers_count":97,"open_issues_count":9,"forks_count":10,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-08-05T09:14:00.536Z","etag":null,"topics":["audio-processing","python","subtitle-conversion","subtitle-to-audio","subtitle-to-speech","subtitle-to-voice","text-to-audio","text-to-speech","tts"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/subtoaudio/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bnsantoso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-07-17T09:53:00.000Z","updated_at":"2024-07-25T00:26:54.000Z","dependencies_parsed_at":"2024-01-13T16:24:49.219Z","dependency_job_id":"95b4b9ed-4fcd-4f36-896e-5e395cb7bd30","html_url":"https://github.com/bnsantoso/sub-to-audio","commit_stats":null,"previous_names":["bnsantoso/sub-to-audio"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnsantoso%2Fsub-to-audio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnsantoso%2Fsub-to-audio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnsantoso%2Fsub-to-audio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnsantoso%2Fsub-to-audio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bnsantoso","download_url":"https://codeload.github.com/bnsantoso/sub-to-audio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225465291,"owners_count":17478518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-processing","python","subtitle-conversion","subtitle-to-audio","subtitle-to-speech","subtitle-to-voice","text-to-audio","text-to-speech","tts"],"created_at":"2024-08-04T09:01:17.006Z","updated_at":"2024-11-20T03:30:52.685Z","avatar_url":"https://github.com/bnsantoso.png","language":"Python","funding_links":["https://ko-fi.com/bnsantoso"],"categories":["Python"],"sub_categories":[],"readme":"#  Subtitle to Audio\nSubtitle to audio, generate audio/speech from any subtitle file using Coqui-ai TTS and synchronize the audio timing according to subtitle time. \n\n**Demo :** [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bnsantoso/sub-to-audio//blob/main/subtitle_to_audio.ipynb)\n\n[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/bnsantoso)\n##  Dependencies\n[ffmpeg](https://ffmpeg.org/), [pydub](https://github.com/jiaaro/pydub), [librosa](https://github.com/librosa/librosa), [coqui-ai TTS](https://github.com/coqui-ai/TTS/), [ffmpeg-python](https://github.com/kkroening/ffmpeg-python)\n\n##  Installation\n\n```bash\npip install TTS\npip install git+https://github.com/bnsantoso/sub-to-audio\n```\n```bash\npip install TTS\npip install subtoaudio\n```\nffmpeg on linux\n```bash\napt-get install ffmpeg\n```\n## Example usage\n\nBasic use is very similiar to [Coqui-ai TTS](https://github.com/coqui-ai/TTS/), you can check their [documentation](https://tts.readthedocs.io/en/latest/inference.html) and the [\u003clang-iso_code\u003e](https://dl.fbaipublicfiles.com/mms/tts/all-tts-languages.html).\n\n**!Note: Use non-overlapping subtitles with an optimal Character per Second / CPS for best result**\n\n**!Note: Use software like aegisub to edit your subtitle**\n\n```python\nfrom subtoaudio import SubToAudio\n\n# list all model\nSubToAudio().coqui_model()\n\n# get model index\nmodel = SubToAudio().coqui_model()[1]\n\n# The code will output 'yoursubtitle.wav' in the current directory.\nsub = SubToAudio(model_name=model)\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle)\n\n# you can choose 1100 different language using fairseq model\nsub = SubToAudio(fairseq_language='\u003clang-iso_code\u003e')\nsubtitle = sub.subtitle(\"yoursubtitle.ass\")\nsub.convert_to_audio(sub_data=subtitle) \n\n# specify model name\nsub = SubToAudio(model_name=\"tts_models/multilingual/multi-dataset/your_tts\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, output_path=\"subtitle.wav\")\n\n# specify model and config path\nsub = SubToAudio(model_path=\"path/to/your/model.pth\" config_path=\"config/path.json\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle)\n\n# speaker=tts.speakers[0] or None if model doesnt have multiple speakers\n# language=tts.languages[0] or None if doesnt have multiple languages\n\n# list speaker\nsub.speakers()\nspeaker1 = sub.speakers()[1]\n\n# list languages\nsub.languages()\nlangu = sub.languages()[0]\n\nsub = SubToAudio(model_name=\"tts_models/multilingual/multi-dataset/your_tts\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, language=langu, speaker=speaker1, output_path=\"subtitle.wav\")\n\n# Save temporary audio to current folder\nsub = SubToAudio(model_name=\"tts_models/multilingual/multi-dataset/your_tts\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, output_path=\"subtitle.wav\", save_temp=True)\n```\n\n## Voice Conversion\n\nTo use voice conversion method, you must pass `voice_conversion:bool` and `speaker_wav:str` paramater on `self.convert_to_audio`. Voice conversion cannot run if your model have multiple speakers.\n\n```python\nfrom subtoaudio import SubToAudio\n\nsub = SubToAudio(fairseq_language=\"eng\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, voice_conversion=True, speaker_wav=\"voice.wav\", language=\"en\")\n```\n\n## Coqui Studio Api\n\nTo use Coqui Studio Api you'll need to configure the COQUI_STUDIO_TOKEN environment variable. \n\n```python\nimport os\n\nos.environ['COQUI_STUDIO_TOKEN'] = # yourapi\n```\n\nAfter your token set you can get coqui studio model, you can follow this name convention `coqui_studio/en/\u003cstudio_speaker_name\u003e/coqui_studio`\n\n```python\nfrom subtoaudio import SubToAudio\n\nsub = SubToAudio(model_name=\"coqui_studio/en/Torcull Diarmuid/coqui_studio\", progress_bar=False)\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, output_path=\"subtitle.wav\", save_temp=True)\n\n# use emotion paramater and speed paramater\nsub.convert_to_audio(sub_data=subtitle, output_path=\"subtitle.wav\", emotion=\"Happy\", speed=1.5)\n```\n\n## Tempo Mode\n\nUse the `tempo_mode` parameter to speed up the audio. There are three tempo modes: \n\n- `tempo_mode=\"all\"` : This accelerates all audio. Use `tempo_speed=float` to specify the speed.\n- `tempo_mode=\"overflow\"` : This accelerates the audio to match the total subtitle duration plus the blank duration before the next subtitle appears. `'tempo_limit'` will limit the speed increase during overflow.\n- `tempo_mode=\"precise\"` : This accelerates the audio to match the duration the subtitle appears.\"\n\n\n```python\nfrom subtoaudio import SubToAudio\n\n# Speed up tempo or speech rate\nsub = SubToAudio(model_name=\"tts_models/de/thorsten/tacotron2-DDC\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"all\", tempo_speed=1.3)\n\n# Change the tempo or speech rate of all audio files , default is 1.2\nsub = SubToAudio(\"tts_models/multilingual/multi-dataset/xtts_v1\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"all\", tempo_speed=1.3)\n\n# Change tempo or speech rate to audio that doesn't match the subtitle duration\nsub = SubToAudio(fairseq_language=\"ind\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"overflow\")\n\n# Limit tempo speed on the overflow mode \nsub = SubToAudio(fairseq_language=\"ind\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"overflow\", tempo_limit=1.2)\n\n# Match audio length to subtitle duration\nsub = SubToAudio(fairseq_language=\"ind\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"precise\")\n```\n\n## Shift Mode\n\n`shift_mode` parameter will shift audio that doesnt match subtitle duration.\n\n- `shift_mode=\"right\"` : Shift audio time to the right and prevent audio overlaping.\n- `shift_mode=\"left\"` : Shift audio to the left and prevent audio overlap, but be cautious of limited space on the left side, as some audio may disappear.\n- `shift_mode=\"interpose\"` : Shift audio to mid position and prevent right and left of audio overlaping. (Note: This mode can be clunky, so use it cautiously.)\n- `shift_mode=\"left-overlap\"` : Shift audio time to the left, allowing overlap.\n- `shift_mode=\"interpose-overlap\"` : Shift audio to mid position, allowing overlap.\n- `shift_limit=int or \"str\"` : limit audio shift, use integer for millisecond or string like `2.5s` for second\n\n```python\nfrom subtoaudio import SubToAudio\n\n# shift mode with limit of 2 second to the right.\n\nsub = SubToAudio(fairseq_language=\"vie\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=sub, tempo_mode=\"overflow\", shift_mode=\"right\", limit_shift=\"2s\")\n\n# shift audio to left position or, time before next subtitle appear\n\nsub = SubToAudio(fairseq_language=\"fra\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=sub, shift_mode=\"left-overlap\")\n\n# shift to left, and limit shift only 1 sec.\nsub = SubToAudio(fairseq_language=\"ind\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=sub, shift_mode=\"left\", shift_limit=1000) # 1000 = 1s\n```\n\n## Bark and Tortoise example\n\n```python\nfrom subtoaudio import SubToAudio\n\n#  Random Speaker will give you weird result when using bark model with SubToAudio\n\n# Bark random\nsub = SubToAudio(\"tts_models/multilingual/multi-dataset/bark\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"overflow\")\n\n# Tortoise random\nsub = SubToAudio(\"tts_models/en/multi-dataset/tortoise-v2\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, shift_mode=\"overflow\", preset=\"fast\")\n\n#  To use voice clone you need voice_dir and speaker paramater\n#  Voice Clone expecting .wav or .npz file inside folder speaker_1\n#  voice/speaker_1/hana.wav or voice/speaker_1/hana.npz\n#  if your speaker folder only have .wav file, it will generate .npz file after you runing it.\n\nsub = SubToAudio(\"tts_models/multilingual/multi-dataset/bark\")\nsubtitle = sub.subtitle(\"yoursubtitle.srt\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"overflow\", voice_dir=\"voice/\",speaker=\"speaker_1\")\n\n# same with bark, the folder structure like this 'voice/speaker2/ron.wav'\nsub = SubToAudio(\"tts_models/en/multi-dataset/tortoise-v2\")\nsubtitle = sub.subtitle(\"yoursubtitle.ass\")\nsub.convert_to_audio(sub_data=subtitle, tempo_mode=\"overflow\", voice_dir=\"voice/\", speaker=\"speaker2\")\n\n```\n\n###  Citation \nEren, G., \u0026 The Coqui TTS Team. (2021). Coqui TTS (Version 1.4) [Computer software]. https://doi.org/10.5281/zenodo.6334862\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbnsantoso%2Fsub-to-audio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbnsantoso%2Fsub-to-audio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbnsantoso%2Fsub-to-audio/lists"}