{"id":15646037,"url":"https://github.com/cyrusvahidi/jtfs-gpu","last_synced_at":"2025-04-30T08:21:09.273Z","repository":{"id":38820712,"uuid":"457296220","full_name":"cyrusvahidi/jtfs-gpu","owner":"cyrusvahidi","description":"code for the paper \"Differentiable Time-Frequency Scattering on GPU\" 🌊","archived":false,"fork":false,"pushed_at":"2023-11-16T16:47:04.000Z","size":1237749,"stargazers_count":55,"open_issues_count":1,"forks_count":4,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-30T14:21:42.411Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://cyrusvahidi.github.io/jtfs-gpu/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cyrusvahidi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-09T09:44:00.000Z","updated_at":"2025-03-03T21:25:51.000Z","dependencies_parsed_at":"2024-10-23T03:01:28.827Z","dependency_job_id":null,"html_url":"https://github.com/cyrusvahidi/jtfs-gpu","commit_stats":{"total_commits":371,"total_committers":9,"mean_commits":41.22222222222222,"dds":0.4824797843665768,"last_synced_commit":"32e2b5de08099a73fb1e47ae110552e21a3e0af8"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyrusvahidi%2Fjtfs-gpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyrusvahidi%2Fjtfs-gpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyrusvahidi%2Fjtfs-gpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyrusvahidi%2Fjtfs-gpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cyrusvahidi","download_url":"https://codeload.github.com/cyrusvahidi/jtfs-gpu/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251666636,"owners_count":21624344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T12:11:06.153Z","updated_at":"2025-04-30T08:21:09.240Z","avatar_url":"https://github.com/cyrusvahidi.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":" \u003cdiv align=\"center\"\u003e    \n  \n# Differentiable Time-Frequency Scattering on GPU 🌊\n\n### source code by\n[Cyrus Vahidi]()\u003csup\u003e2\u003c/sup\u003e \u003cbr\u003e\n[Changhong Wang]()\u003csup\u003e1\u003c/sup\u003e, [Han Han]()\u003csup\u003e1\u003c/sup\u003e \n[Vincent Lostanlen]()\u003csup\u003e1\u003c/sup\u003e \u003cbr\u003e\n[John Muradeli]() \u003cbr\u003e\nLS2N/CNRS Centrale Nantes\u003csup\u003e1\u003c/sup\u003e  Queen Mary University of London\u003csup\u003e2\u003c/sup\u003e\n\n\n  [![Paper](http://img.shields.io/badge/paper-arxiv.2204.08269-B31B1B.svg)](https://arxiv.org/abs/2204.08269)\n  \u003c!--\n  ARXIV   \n  [![Paper](http://img.shields.io/badge/arxiv-math.co:1480.1111-B31B1B.svg)](https://www.nature.com/articles/nature14539)\n  --\u003e\n\n  \u003c!--  \n  Conference   \n  --\u003e   \n  \u003c/div\u003e\n  \n  \n![Accompanying webpage 🌐](https://cyrusvahidi.github.io/jtfs-gpu/) \u003cbr\u003e\n![Kymatio: open-source wavelet scattering in Python ‍💻](https://github.com/kymatio/kymatio/)\n  \nMany thanks to all open-source contributors to ![Kymatio](https://github.com/kymatio/kymatio) and its dependencies.\n   \nThis repository contains code to replicate the paper \"Differentiable Time-Frequency Scattering on GPU\" (published at DAFx 2022, best paper award).\n\nTime-frequency scattering is available in ![Kymatio](https://github.com/kymatio/kymatio/) in beta and will be released in v0.4. To use this implementation, install Kymatio from source. To replicate the results in this paper, follow the installation instructions below.\n\nWe assess Time-Frequency Scattering in Kymatio for 3 machine listening research applications:\n\n* unsupervised manifold learning of spectrotemporal modulations\n* hybrid jtfs + convnet supervised musical instrument classification\n* texture resynthesis \n\n\u003c!-- We also provide scale-rate visualizations:\n\n\u003cimg src=\"https://user-images.githubusercontent.com/16495490/163857080-9ae52cad-9202-4fb8-a1f5-a7d008f19073.png\" alt=\"signal\" width=\"800\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/16495490/163851994-b35772b0-5f73-4eef-8417-26ad02bbb65c.png\" alt=\"scale-rate\" width=\"750\"\u003e\n --\u003e\n\n## How to run   \n\nFirst, install dependencies\n\n### Installation\n\n```bash\n# clone project   \ngit clone https://github.com/cyrusvahidi/jtfs-gpu\n\n# install project   \ncd jtfs-gpu\npip install -e .\npip install -r requirements.txt\n\n# install kymatio from source\ncd kymatio\npython setup.py develop\n```   \n\n* The JTFS algorithm source code to replicate the paper can be found ![here](https://github.com/overLordGoldDragon/wavespin/tree/dafx2022-jtfs)\n* The latest version of JTFS can be installed directly from the ![Kymatio](https://github.com/kymatio/kymatio/) source\n\n### ConvNet Classifier\n\n[Download Medley-solos-DB](https://zenodo.org/record/3464194)\n\n#### Extract Scattering Features\n``` bash\npython scripts/process_msdb_features.py --data_dir \u003cyour_msdb_dir\u003e --feature \u003cfeature_to_extract\u003e\n```\n\n#### Configure gin\nIn `/scripts/gin/config.gin` set `MSDB_DATA_DIR` and `MSDB_CSV` according to the absolute path of your MSDB download.\n\n#### Run training\n\n``` bash\npython scripts/train_cnn.py\n```\n\n### Isomap Visualizations \u0026 K-NN Regression\n``` bash\npython scripts/isomap.py\n```\nsee the output in `/img`\n\n### Scale-Rate Visualizations and Resynthesis\n```\ncd notebooks\njupyter notebook\n```\nSee `Scale-Rate Visualization.ipynb` and `Resynthesis results.ipynb`\n\n### Notebook Guide\n\n#### Scale-Rate Visualisations\n\nH E L L O\n\n#### Synthetic amplitude-modulated chirp dataset\n* Factors of variation: \n * $f_c$ carrier frequency\n * $f_m$ amplitude modulation frequency\n * $\\gamma$ chirp rate\n\n#### Manifold Embedding of the Nearest Neighbour Graph\n* MFCCs\n* Time Scattering\n* Time-Frequency Scattering\n* Open-L3\n* Spectrotemporal Receptive Field (STRF)\n\n#### K-NN regression of synthesizer parameters\n\n#### 2-D CNN classifier\n\n#### Differentiable Resynthesis \n\n### Citation   \n```\n@article{muradeli2022differentiable,\n  title={Differentiable Time-Frequency Scattering in Kymatio},\n  author={John Muradeli, Cyrus Vahidi, Changhong Wang, Han Han, Vincent Lostanlen, Mathieu Lagrange, George Fazekas},\n  journal={arXiv preprint arXiv:2204.08269},\n  year={2022}\n}\n```   \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyrusvahidi%2Fjtfs-gpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyrusvahidi%2Fjtfs-gpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyrusvahidi%2Fjtfs-gpu/lists"}