https://github.com/sony/clipsep
https://github.com/sony/clipsep
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/sony/clipsep
- Owner: sony
- License: mit
- Created: 2023-01-24T03:45:58.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-02-21T14:25:02.000Z (over 2 years ago)
- Last Synced: 2025-04-19T16:52:34.578Z (6 months ago)
- Language: Python
- Size: 42.9 MB
- Stars: 40
- Watchers: 2
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
## CLIPSep - Official PyTorch Implementation
> **CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos**
> [Hao-Wen Dong](https://salu133445.github.io/), [Naoya Takahashi](https://kotonaoya.wixsite.com/naoya-takahashi)*, [Yuki Mitsufuji](https://www.yukimitsufuji.com/), [Julian McAuley](https://cseweb.ucsd.edu/~jmcauley/), [Taylor Berg-Kirkpatrick](https://cseweb.ucsd.edu/~tberg/)
> In ICLR 2023. (\* corresponding author)
> [arXiv](https://arxiv.org/abs/2212.07065) | [OpenReview](https://openreview.net/forum?id=H-T3F0dMbyj) | [Demo](https://sony.github.io/CLIPSep)
## Orgnization
+ [`music`](https://github.com/sony/CLIPSep/tree/main/music): Code for downloading and preprocessing the MUSIC dataset.
+ [`vggsound`](https://github.com/sony/CLIPSep/tree/main/vggsound): Code for downloading and preprocessing the VGGSound dataset.
+ [`clipsep`](https://github.com/sony/CLIPSep/tree/main/clipsep): Code for training and experimenting with the CLIPSep model.
## Citation
If you find this work useful for your research, please cite our paper:
```
@inproceedings{dong2023clipsep,
title={CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos},
author={Hao-Wen Dong and Naoya Takahashi and Yuki Mitsufuji and Julian McAuley and Taylor Berg-Kirkpatrick},
booktitle={Proceedings of International Conference on Learning Representations (ICLR)},
year={2023}
}
```