Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/eigenpunk/ComfyUI-audio
some generative audio tools for ComfyUI
https://github.com/eigenpunk/ComfyUI-audio
audio-processing comfyui musicgen text-to-music text-to-speech tortoise-tts
Last synced: 4 days ago
JSON representation
some generative audio tools for ComfyUI
- Host: GitHub
- URL: https://github.com/eigenpunk/ComfyUI-audio
- Owner: eigenpunk
- License: gpl-3.0
- Created: 2023-12-06T13:24:00.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-03-03T21:14:14.000Z (4 months ago)
- Last Synced: 2024-05-15T09:44:34.697Z (about 2 months ago)
- Topics: audio-processing, comfyui, musicgen, text-to-music, text-to-speech, tortoise-tts
- Language: Python
- Homepage:
- Size: 107 KB
- Stars: 52
- Watchers: 3
- Forks: 14
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- my-awesome-stars - eigenpunk/ComfyUI-audio - some generative audio tools for ComfyUI (Python)
README
# ComfyUI-audio
generative audio tools for ComfyUI. highly experimental—expect things to break and/or change frequently or not at all.
**NOTE**: for the foreseeable future, i will be unable to continue working on this extension. please consider forking this repository!
## features
- [tacotron2 text-to-speech](https://github.com/NVIDIA/tacotron2)
- uses justinjohn0306's forks of [tacotron2](https://github.com/justinjohn0306/TTS-TT2/) and [hifi-gan](https://github.com/justinjohn0306/hifi-gan/)
- [musicgen text-to-music + audiogen text-to-sound](https://facebookresearch.github.io/audiocraft/docs/MUSICGEN.html)
- audiocraft and transformers implementations
- supports audio continuation, unconditional generation
- [tortoise text-to-speech](https://github.com/neonbjb/tortoise-tts)
- [vall-e x text-to-speech](https://github.com/Plachtaa/VALL-E-X)
- uses [korakoe's fork](https://github.com/korakoe/VALL-E-X)
- [voicefixer](https://github.com/voicefixer/voicefixer)
- audio utility nodes
- save audio, convert audio
## installation
```shell
# TORCH_CUDA_INDEX_URL=https://download.pytorch.org/whl/cu118 # for cuda 11.8
TORCH_CUDA_INDEX_URL=https://download.pytorch.org/whl/cu121 # for cuda 12.1
cd ComfyUI/custom_nodes
git clone https://github.com/eigenpunk/ComfyUI-audio
cd ComfyUI-audio
# for linux
pip install -r requirements.txt --extra-index-url $TORCH_CUDA_INDEX_URL
# for windows
pip install -r requirements_windows.txt --extra-index-url $TORCH_CUDA_INDEX_URL
```
this extension is developed and tested on a Linux-based OS. i've not yet been able to get the extension fully working on Windows, so
expect some difficulty if that is your platform. i've not tested the extension on macOS at all.
## would be nice to have maybe
- audio uploads
- audio previews
- prompt weights for text-to-music/audio
- stereo musicgen
- multi-band diffusion
- more/faster tts model support
- [vits](https://huggingface.co/docs/transformers/model_doc/vits)?
- ~~[tacotron2](https://github.com/NVIDIA/tacotron2)~~
- ~~[vall-e x](https://github.com/Plachtaa/VALL-E-X)~~
- ???
- split generator nodes by model stages
- e.g. tortoise:
- autoregressor
- clvp/cvvp
- spectrogram diffusion
- e.g. musicgen:
- t5 text encode
- encodec audio encode
- generate with decoder
- more audio generation models
- magnet, etc
- demucs
- ~~audiogen~~
*NOTE*: this work is solely a personal project; its development is not supported/sponsored by any past/present employer or any other external organization.