{"id":13434418,"url":"https://github.com/archinetai/audio-ai-timeline","last_synced_at":"2026-02-05T16:31:15.374Z","repository":{"id":65559084,"uuid":"594679568","full_name":"archinetai/audio-ai-timeline","owner":"archinetai","description":"A timeline of the latest AI models for audio generation, starting in 2023!","archived":false,"fork":false,"pushed_at":"2024-01-04T01:15:07.000Z","size":71,"stargazers_count":1905,"open_issues_count":1,"forks_count":70,"subscribers_count":165,"default_branch":"main","last_synced_at":"2025-07-07T19:47:16.763Z","etag":null,"topics":["artificial-intelligence","audio-generation","machine-learning"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/archinetai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-01-29T09:58:56.000Z","updated_at":"2025-07-06T21:33:06.000Z","dependencies_parsed_at":"2023-02-20T19:30:19.026Z","dependency_job_id":"817fc518-d39d-43cd-9570-05281a9876f8","html_url":"https://github.com/archinetai/audio-ai-timeline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/archinetai/audio-ai-timeline","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/archinetai%2Faudio-ai-timeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/archinetai%2Faudio-ai-timeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/archinetai%2Faudio-ai-timeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/archinetai%2Faudio-ai-timeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/archinetai","download_url":"https://codeload.github.com/archinetai/audio-ai-timeline/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/archinetai%2Faudio-ai-timeline/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29125814,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T14:05:12.718Z","status":"ssl_error","status_checked_at":"2026-02-05T14:03:53.078Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","audio-generation","machine-learning"],"created_at":"2024-07-31T02:01:56.229Z","updated_at":"2026-02-05T16:31:15.356Z","avatar_url":"https://github.com/archinetai.png","language":null,"funding_links":[],"categories":["Others","Image Segmentation"],"sub_categories":["Creative Uses of Generative AI Image Synthesis Tools"],"readme":"# Audio AI Timeline\n\nHere we will keep track of the latest AI models for waveform based audio generation, starting in 2023!\n\n## 2023\n\n| Date | Release [Samples] | Paper | Code | Trained Model |\n| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| 14.11 | [Mustango: Toward Controllable Text-to-Music Generation](https://amaai-lab.github.io/mustango/) | [arXiv](https://arxiv.org/abs/2311.08355) | [GitHub](https://github.com/AMAAI-Lab/mustango) | [Hugging Face](https://huggingface.co/declare-lab/mustango) |\n| 13.11 | [Music ControlNet: Multiple Time-varying Controls for Music Generation](https://musiccontrolnet.github.io/web/) | [arXiv](https://arxiv.org/abs/2311.07069) | - | - |\n| 02.11 | [E3 TTS: Easy End-to-End Diffusion-based Text to Speech](https://e3tts.github.io/) | [arXiv](https://arxiv.org/abs/2311.00945) | - | - |\n| 01.10 | [UniAudio: An Audio Foundation Model Toward Universal Audio Generation](http://dongchaoyang.top/UniAudio_demo/) | [arXiv](https://arxiv.org/abs/2310.00704) | [GitHub](https://github.com/yangdongchao/UniAudio) | - |\n| 24.09 | [VoiceLDM: Text-to-Speech with Environmental Context](https://voiceldm.github.io/) | [arXiv](https://arxiv.org/abs/2309.13664) | [GitHub](https://github.com/glory20h/VoiceLDM) | - |\n| 05.09 | [PromptTTS 2: Describing and Generating Voices with Text Prompt](https://speechresearch.github.io/prompttts2/) | [arXiv](https://arxiv.org/abs/2309.02285) | - | - |\n| 14.08 | [SpeechX: Neural Codec Language Model as a Versatile Speech Transformer](https://www.microsoft.com/en-us/research/project/speechx/) | [arXiv](https://arxiv.org/abs/2308.06873) | - | - |\n| 10.08 | [AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining](https://audioldm.github.io/audioldm2/) | [arXiv](https://arxiv.org/abs/2308.05734) | [GitHub](https://github.com/haoheliu/audioldm2) | [Hugging Face](https://huggingface.co/spaces/haoheliu/audioldm2-text2audio-text2music) |\n| 09.08 | [JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models](https://www.futureverse.com/research/jen/demos/jen1) | [arXiv](https://arxiv.org/abs/2308.04729) | - | - |\n| 03.08 | [MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies](https://musicldm.github.io/) | [arXiv](https://arxiv.org/abs/2308.01546) | [GitHub](https://github.com/RetroCirce/MusicLDM/) | - |\n| 14.07 | [Mega-TTS 2: Zero-Shot Text-to-Speech with Arbitrary Length Speech Prompts](https://mega-tts.github.io/mega2_demo/) | [arXiv](https://arxiv.org/abs/2307.07218) | - | - |\n| 10.07 | [VampNet: Music Generation via Masked Acoustic Token Modeling](https://hugo-does-things.notion.site/VampNet-Music-Generation-via-Masked-Acoustic-Token-Modeling-e37aabd0d5f1493aa42c5711d0764b33) | [arXiv](https://arxiv.org/abs/2307.04686) | [GitHub](https://github.com/hugofloresgarcia/vampnet) | - |\n| 22.06 | [AudioPaLM: A Large Language Model That Can Speak and Listen](https://google-research.github.io/seanet/audiopalm/examples/) | [arXiv](https://arxiv.org/abs//2306.12925) | - | - |\n| 19.06 | [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://voicebox.metademolab.com/) | [PDF](https://scontent-lga3-2.xx.fbcdn.net/v/t39.8562-6/354636794_599417672291955_3799385851435258804_n.pdf?_nc_cat=101\u0026ccb=1-7\u0026_nc_sid=ad8a9d\u0026_nc_ohc=bN1S0esWehwAX_22ORV\u0026_nc_ht=scontent-lga3-2.xx\u0026oh=00_AfCtouRXIvwDx10qPVMNkq_4xMTVOQUrfmyYQW--9cIoWg\u0026oe=64947BF1) | [GitHub](https://github.com/SpeechifyInc/Meta-voicebox) | - |\n| 08.06 | [MusicGen: Simple and Controllable Music Generation](https://ai.honu.io/papers/musicgen/) | [arXiv](https://arxiv.org/abs/2306.05284) | [GitHub](https://github.com/facebookresearch/audiocraft) | [Hugging Face](https://huggingface.co/spaces/facebook/MusicGen) [Colab](https://colab.research.google.com/drive/1fxGqfg96RBUvGxZ1XXN07s3DthrKUl4-?usp=sharing) |\n| 06.06 | [Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias](https://mega-tts.github.io/demo-page/) | [arXiv](https://arxiv.org/abs/2306.03509) | - | - |\n| 01.06 | [Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis](https://charactr-platform.github.io/vocos/) | [arXiv](https://arxiv.org/abs/2306.00814) | [GitHub](https://github.com/charactr-platform/vocos) | - |\n| 29.05 | [Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation](https://make-an-audio-2.github.io/) | [arXiv](https://arxiv.org/abs/2305.18474) | - | - |\n| 25.05 | [MeLoDy: Efficient Neural Music Generation](https://efficient-melody.github.io/) | [arXiv](https://arxiv.org/abs/2305.15719) | - | - |\n| 18.05 | [CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-training](https://clapspeech.github.io/) | [arXiv](https://arxiv.org/abs/2305.10763) | - | - |\n| 18.05 | [SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities](https://0nutation.github.io/SpeechGPT.github.io/) | [arXiv](https://arxiv.org/abs/2305.11000) | [GitHub](https://github.com/0nutation/SpeechGPT) | - |\n| 16.05 | [SoundStorm: Efficient Parallel Audio Generation](https://google-research.github.io/seanet/soundstorm/examples/) | [arXiv](https://arxiv.org/abs/2305.09636) | [GitHub (unofficial)](https://github.com/lucidrains/soundstorm-pytorch) | - |\n| 03.05 | [Diverse and Vivid Sound Generation from Text Descriptions](https://ligw1998.github.io/audiogeneration.html) | [arXiv](https://arxiv.org/abs/2305.01980) | - | - |\n| 02.05 | [Long-Term Rhythmic Video Soundtracker](https://justinyuu.github.io/LORIS/) | [arXiv](https://arxiv.org/abs/2305.01319) | [GitHub](https://github.com/OpenGVLab/LORIS) | - |\n| 24.04 | [TANGO: Text-to-Audio generation using instruction tuned LLM and Latent Diffusion Model](https://tango-web.github.io/) | [PDF](https://openreview.net/pdf?id=1Sn2WqLku1e) | [GitHub](https://github.com/declare-lab/tango) | [Hugging Face](https://huggingface.co/declare-lab/tango) |\n| 18.04 | [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://speechresearch.github.io/naturalspeech2/) | [arXiv](https://arxiv.org/abs/2304.09116) | [GitHub (unofficial)](https://github.com/lucidrains/naturalspeech2-pytorch) | - |\n| 10.04 | [Bark: Text-Prompted Generative Audio Model](https://github.com/suno-ai/bark) | - | [GitHub](https://github.com/suno-ai/bark) | [Hugging Face](https://huggingface.co/spaces/suno/bark) [Colab](https://colab.research.google.com/drive/1eJfA2XUa-mXwdMy7DoYKVYHI1iTd9Vkt?usp=sharing) |\n| 03.04 | [AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models](https://audit-demo.github.io/) | [arXiv](https://arxiv.org/abs/2304.00830) | - | - |\n| 08.03 | [VALL-E X: Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling](https://www.microsoft.com/en-us/research/project/vall-e-x/) | [arXiv](https://arxiv.org/abs/2303.03926) | - | - |\n| 27.02 | [I Hear Your True Colors: Image Guided Audio Generation](https://pages.cs.huji.ac.il/adiyoss-lab/im2wav/) | [arXiv](https://arxiv.org/abs/2211.03089) | [GitHub](https://github.com/RoySheffer/im2wav) | - |\n| 08.02 | [Noise2Music: Text-conditioned Music Generation with Diffusion Models](https://google-research.github.io/noise2music/) | [arXiv](https://arxiv.org/abs/2302.03917) | - | - |\n| 04.02 | [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://gladia-research-group.github.io/multi-source-diffusion-models/) | [arXiv](https://arxiv.org/abs/2302.02257) | [GitHub](https://github.com/gladia-research-group/multi-source-diffusion-models) | - |\n| 30.01 | [SingSong: Generating musical accompaniments from singing](https://storage.googleapis.com/sing-song/index.html) | [arXiv](https://arxiv.org/abs/2301.12662) | - | - |\n| 30.01 | [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://audioldm.github.io/) | [arXiv](https://arxiv.org/abs/2301.12503) | [GitHub](https://github.com/haoheliu/AudioLDM) | [Hugging Face](https://huggingface.co/spaces/haoheliu/audioldm-text-to-audio-generation) |\n| 30.01 | [Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion](https://anonymous0.notion.site/Mo-sai-Text-to-Audio-with-Long-Context-Latent-Diffusion-b43dbc71caf94b5898f9e8de714ab5dc) | [arXiv](https://arxiv.org/abs/2301.11757) | [GitHub](https://github.com/archinetai/audio-diffusion-pytorch) | - |\n| 29.01 | [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://text-to-audio.github.io/) | [PDF](https://text-to-audio.github.io/paper.pdf) | - | - |\n| 28.01 | [Noise2Music](https://noise2music.github.io/) | - | - | - |\n| 27.01 | [RAVE2](https://twitter.com/antoine_caillon/status/1618959533065535491?s=20\u0026t=jMkPWBFuAH19HI9m5Sklmg) [[Samples RAVE1](https://anonymous84654.github.io/RAVE_anonymous/)] | [arXiv](https://arxiv.org/abs/2111.05011) | [GitHub](https://github.com/acids-ircam/RAVE) | - |\n| 26.01 | [MusicLM: Generating Music From Text](https://google-research.github.io/seanet/musiclm/examples/) | [arXiv](https://arxiv.org/abs/2301.11325) | [GitHub (unofficial)](https://github.com/lucidrains/musiclm-pytorch) | - |\n| 18.01 | [Msanii: High Fidelity Music Synthesis on a Shoestring Budget](https://kinyugo.github.io/msanii-demo/) | [arXiv](https://arxiv.org/abs/2301.06468) | [GitHub](https://github.com/Kinyugo/msanii) | [Hugging Face](https://huggingface.co/spaces/kinyugo/msanii) [Colab](https://colab.research.google.com/github/Kinyugo/msanii/blob/main/notebooks/msanii_demo.ipynb) |\n| 16.01 | [ArchiSound: Audio Generation with Diffusion](https://flavioschneider.notion.site/Audio-Generation-with-Diffusion-c4f29f39048d4f03a23da13078a44cdb) | [arXiv](https://arxiv.org/abs/2301.13267) | [GitHub](https://github.com/archinetai/audio-diffusion-pytorch) | - |\n| 05.01 | [VALL-E: Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://www.microsoft.com/en-us/research/project/vall-e-x/) | [arXiv](https://arxiv.org/abs/2301.02111) | [GitHub (unofficial)](https://github.com/lifeiteng/vall-e) [(demo)](https://lifeiteng.github.io/valle/index.html) | - |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchinetai%2Faudio-ai-timeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farchinetai%2Faudio-ai-timeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farchinetai%2Faudio-ai-timeline/lists"}