{"id":13861691,"url":"https://github.com/hubertsiuzdak/snac","last_synced_at":"2025-07-14T09:33:06.411Z","repository":{"id":223409819,"uuid":"760223932","full_name":"hubertsiuzdak/snac","owner":"hubertsiuzdak","description":"Multi-Scale Neural Audio Codec (SNAC) compresses audio into discrete codes at a low bitrate","archived":false,"fork":false,"pushed_at":"2024-10-22T16:14:59.000Z","size":5250,"stargazers_count":437,"open_issues_count":13,"forks_count":26,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-11-16T18:47:21.180Z","etag":null,"topics":["audio","audio-codec","deep-learning"],"latest_commit_sha":null,"homepage":"https://hubertsiuzdak.github.io/snac/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hubertsiuzdak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-20T02:33:25.000Z","updated_at":"2024-11-14T07:21:54.000Z","dependencies_parsed_at":"2024-10-23T00:03:48.799Z","dependency_job_id":null,"html_url":"https://github.com/hubertsiuzdak/snac","commit_stats":null,"previous_names":["hubertsiuzdak/snac"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hubertsiuzdak%2Fsnac","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hubertsiuzdak%2Fsnac/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hubertsiuzdak%2Fsnac/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hubertsiuzdak%2Fsnac/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hubertsiuzdak","download_url":"https://codeload.github.com/hubertsiuzdak/snac/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225968854,"owners_count":17553147,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","audio-codec","deep-learning"],"created_at":"2024-08-05T06:01:28.143Z","updated_at":"2024-11-22T21:31:00.316Z","avatar_url":"https://github.com/hubertsiuzdak.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# SNAC 🍿\n\nMulti-**S**cale **N**eural **A**udio **C**odec (SNAC) compresses audio into discrete codes at a low bitrate. For more information, read the paper: https://arxiv.org/abs/2410.14411\n\n| 🎸 Music samples                                                                                         | 🗣️ Speech samples                                                                                       |\n|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|\n| \u003cvideo src='https://github.com/hubertsiuzdak/snac/assets/35269911/e8adac68-d3f1-4fc1-8cf9-f48d9bcd95ed'\u003e | \u003cvideo src='https://github.com/hubertsiuzdak/snac/assets/35269911/65ac2547-c711-49d4-8a5d-64d52e6d6ba1'\u003e |\n\n🎧 More audio samples available at https://hubertsiuzdak.github.io/snac/\n\n## Overview\n\nSNAC encodes audio into hierarchical tokens similarly to SoundStream, EnCodec, and DAC (see the image\non the left). However, SNAC introduces a simple change where coarse tokens are sampled less frequently,\ncovering a broader time span (see the image on the right).\n\nThis can not only save on bitrate, but more importantly this might be very useful for language modeling approaches to\naudio generation. E.g. with coarse tokens of ~10 Hz and a context window of 2048 you can effectively model a\nconsistent structure of an audio track for ~3 minutes.\n\n![snac.png](img%2Fsnac.png)\n\n## Pretrained models\n\nCurrently, all models support only single audio channel (mono).\n\n| Model                                                                       | Bitrate   | Sample Rate | Params | Recommended use case     | \n|-----------------------------------------------------------------------------|-----------|-------------|--------|--------------------------|\n| [hubertsiuzdak/snac_24khz](https://huggingface.co/hubertsiuzdak/snac_24khz) | 0.98 kbps | 24 kHz      | 19.8 M | 🗣️ Speech               | \n| [hubertsiuzdak/snac_32khz](https://huggingface.co/hubertsiuzdak/snac_32khz) | 1.9 kbps  | 32 kHz      | 54.5 M | 🎸 Music / Sound Effects | \n| [hubertsiuzdak/snac_44khz](https://huggingface.co/hubertsiuzdak/snac_44khz) | 2.6 kbps  | 44 kHz      | 54.5 M | 🎸 Music / Sound Effects |\n\n## Usage\n\nInstall it using:\n\n```bash\npip install snac\n```\n\nTo encode (and decode) audio with SNAC in Python, use the following code:\n\n```python\nimport torch\nfrom snac import SNAC\n\nmodel = SNAC.from_pretrained(\"hubertsiuzdak/snac_32khz\").eval().cuda()\naudio = torch.randn(1, 1, 32000).cuda()  # placeholder for actual audio with shape (B, 1, T)\n\nwith torch.inference_mode():\n    codes = model.encode(audio)\n    audio_hat = model.decode(codes)\n```\n\nYou can also encode and reconstruct in a single call:\n\n```python\nwith torch.inference_mode():\n    audio_hat, codes = model(audio)\n```\n\n⚠️ Note that `codes` is a list of token sequences of variable lengths, each corresponding to a different temporal\nresolution.\n\n```\n\u003e\u003e\u003e [code.shape[1] for code in codes]\n[12, 24, 48, 96]\n```\n\n## Acknowledgements\n\nModule definitions are adapted from the [Descript Audio Codec](https://github.com/descriptinc/descript-audio-codec).\n\n## Citation\n\nIf this code contributes to your research, please cite our work:\n\n```\n@inproceedings{siuzdak2024snac,\n  title={SNAC: Multi-Scale Neural Audio Codec},\n  author={Siuzdak, Hubert and Gr{\\\"o}tschla, Florian and Lanzend{\\\"o}rfer, Luca A},\n  booktitle={Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation},\n  year={2024}\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhubertsiuzdak%2Fsnac","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhubertsiuzdak%2Fsnac","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhubertsiuzdak%2Fsnac/lists"}