{"id":20566103,"url":"https://github.com/gregogiudici/knowledge-distillation_ddsp-decoder","last_synced_at":"2026-05-06T00:06:57.888Z","repository":{"id":199159166,"uuid":"702252619","full_name":"gregogiudici/Knowledge-Distillation_DDSP-Decoder","owner":"gregogiudici","description":"Knowledge Distillation of different DDSP Decoders (GRU, TCN, S4)","archived":false,"fork":false,"pushed_at":"2023-10-09T12:27:49.000Z","size":196,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-06T08:57:36.628Z","etag":null,"topics":["ddsp","gru","s4","tcn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gregogiudici.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-08T23:44:00.000Z","updated_at":"2024-12-21T11:19:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"719c6260-6278-487c-95b3-bd3190f9a6c6","html_url":"https://github.com/gregogiudici/Knowledge-Distillation_DDSP-Decoder","commit_stats":null,"previous_names":["gregogiudici/knowledge-distillation_ddsp-decoder"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gregogiudici/Knowledge-Distillation_DDSP-Decoder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregogiudici%2FKnowledge-Distillation_DDSP-Decoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregogiudici%2FKnowledge-Distillation_DDSP-Decoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregogiudici%2FKnowledge-Distillation_DDSP-Decoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregogiudici%2FKnowledge-Distillation_DDSP-Decoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gregogiudici","download_url":"https://codeload.github.com/gregogiudici/Knowledge-Distillation_DDSP-Decoder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gregogiudici%2FKnowledge-Distillation_DDSP-Decoder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32672688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ddsp","gru","s4","tcn"],"created_at":"2024-11-16T04:40:01.181Z","updated_at":"2026-05-06T00:06:57.871Z","avatar_url":"https://github.com/gregogiudici.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n______________________________________________________________________\n\n\u003cdiv align=\"center\"\u003e\n\n# Knowledge Distillation of neural models for audio signal generation\n\u003ca href=\"https://pytorch.org/get-started/locally/\"\u003e\u003cimg alt=\"PyTorch\" src=\"https://img.shields.io/badge/PyTorch-ee4c2c?logo=pytorch\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://pytorchlightning.ai/\"\u003e\u003cimg alt=\"Lightning\" src=\"https://img.shields.io/badge/-Lightning-792ee5?logo=pytorchlightning\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003ca href=\"https://hydra.cc/\"\u003e\u003cimg alt=\"Config: Hydra\" src=\"https://img.shields.io/badge/Config-Hydra-89b8cd\"\u003e\u003c/a\u003e\n\u003ca href=\"https://magenta.tensorflow.org/ddsp\"\u003e\u003cimg alt=\"DDSP\" src=\"https://img.shields.io/badge/DDSP-Magenta-792ee5\"\u003e\u003c/a\u003e\n\n\u003ccenter\u003e\n\u003cimg src=\"img/distillation_graph.png\"\"\u003e\n\u003c/center\u003e\n\u003cbr\u003e\n\n\u003c/div\u003e\n\n## Description\nThis repository is part of my master's degreee thesis.\nMain goal of the thesis is to analyze the effects of Knowledge Distillation (KD) on different DDSP decoders for audio signal generation.\n\nThree teacher models, called \"*large*\" models (4.5M parameters), have been implemented, each corresponding to a DDSP decoder controlling a Harmonic-plus-Noise synthesizer: \n- GRUdecoder-L: Gated Recurrent Unit (GRU)\n- TCNdecoder-L: Temporal Convolution Network (TCN)\n- S4decoder-L: [Structured State Space Sequence](https://github.com/HazyResearch/state-spaces/tree/main) (S4)\n\nTwo types of experiments were performed in order to analyze the effectiveness of the Knowledge Distillation, both when the student corresponds to a reduced version of the teacher, and when it corresponds to a model with different architecture.\n\n**Student with same architecture as teacher**:\n- \"*medium*\" (400K parameters): GRUdecoder-M, TCNdecoder-M, S4decoder-M\n- \"*small*\" (40K parameters): GRUdecoder-S, TCNdecoder-S, S4decoder-S\n\n**Student with different architecture:**\n- [DDX7](https://github.com/fcaspe/ddx7/tree/main) (400K parameters)\n\n\n\n## Installation\n\n#### Conda + Pip\n\n```bash\n# Clone project\ngit clone git@github.com:gregogiudici/Knowledge-Distillation_DDSP-Decoder.git\ncd Knowledge-Distillation_DDSP-Decoder\n\n# Create conda environment\nconda create -n myenv python=3.9\nconda activate myenv\nconda install pytorch==1.13.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia\n# Some care may be needed to make compatible the library versions of torch, torchaudio, etc\n\n# Install requirements\npip install -r requirements.txt\n\n# For S4decoder: compile CUDA extension for cauchy multiplication \n# (not required but recommended)\ncd extensions/cauchy/ \npython setup.py install\n```\n#### ATTENTION:\nRemember to \u003cins\u003econfigure\u003c/ins\u003e the development environment (**.env**) file: this file is used to pass environment variables for the proper operation of this project.\n\n## Dataset generation\n\nWe used the [URMP](https://labsites.rochester.edu/air/projects/URMP.html) dataset to train and test the models.\nAdditional test files can be aggregated and used for resynthesis tasks.\nPlease check the `dataset` directory for advanced options to process and build a dataset.\n\n**Quick start** - will extract and process violin, flute, and trumpet data using [`torchcrepe`](https://github.com/maxrmorrison/torchcrepe).\n\n```bash\ncd dataset\npython create_data.py urmp.source_folder=/path/to/URMP/Dataset\n```\n\n## Quick-start\n### Train model without Knowledge Distillation\n\nTrain model with default configuration\n\n```bash\n# Train with accelerator=auto (CPU, GPU, DDP, etc)\npython src/train.py\n\n# Train on CPU\npython src/train.py trainer=cpu\n\n# Train on GPU\npython src/train.py trainer=gpu\n```\nTrain different model (sample-rate=16kHz \u0026 frame-rate=250Hz)\n```bash\n# Train GRUdecoder (default) with Flute (default)\n# dataset: sr=16kHz, fr=250\npython src/train.py #model=gru/large_gru_decoder\n\n# Train TCNdecoder with Flute (default)\npython src/train.py model=tcn/medium_tcn_decoder\n\n# Train DDX7 with Flute (default)\npython src/train.py model=ddx7/tcnres_f0ld_fmflt.yaml\n```\nTrain model with different instrument (sample-rate=16kHz \u0026 frame-rate=250Hz)\n\n```bash\n# Train GRUdecoder (default) with Flute (default)\n# dataset: sr=16kHz, fr=250\npython src/train.py #data=flute_16000_250\n\n# Train GRUdecoder (default) with Violin\npython src/train.py data=violin_16000_250 \n\n# Train GRUdecoder (default) with Trumpet\npython src/train.py data=trumpet_16000_250\n```\n\nYou can override any parameter from command-line like this\n\n```bash\npython src/train.py trainer.max_epochs=5000 data.batch_size=64 data.instrument=violin_16000_250 \n```\n\n## Knowledge Distillation: experiments\nChoose experiment configuration from [configs/experiment/](configs/experiment/)\n\n### 1. Train TEACHER \n\n```bash\n# Experiment: GRUdecoder-L\n#       dimension: large\n#       model: GRUdecoder\n#       instrument: flute\npython src/train.py experiment=gru/large_gru_flute\n```\n### 2. Add TEACHER checkpoint to your .env file\nThis is necessary for the Knowledge Distillation since you have to indicate where the pre-trained teacher model is located.\n```bash\nLARGE_GRU_FLUTE_16000_250=\"${ROOT}/logs/large/runs/1999-12-31_00-00-00/checkpoints/epoch_000.ckpt\"\n\n```\n\n### 3. Train STUDENT without Knowledge Distillation\n```bash\n# Experiment: GRUdecoder-M\n#       dimension: medium\n#       model: GRUdecoder\n#       instrument: flute\npython src/train.py experiment=gru/medium_gru_flute\n\n# Experiment: DDX7\n#       model: DDX7 with fmflt patch\n#       instrument: flute\npython src/train.py experiment=ddx7/ddx7_fmflt_flute\n```\n### 4. Train STUDENT with Knowledge Distillation\n```bash\n# Experiment: KD-GRUdecoder-M\n#       dimension: medium\n#       model: GRUdecoder\n#       instrument: flute\npython src/distillation.py experiment=kd/large2medium/gru2gru_flute\n\n# Experiment: KD-DDX7 (GRU)\n#       model: DDX7 with fmflt patch\n#       instrument: flute\npython src/distillation.py experiment=kd/hpn2ddx7/gru2ddx7_flute\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgregogiudici%2Fknowledge-distillation_ddsp-decoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgregogiudici%2Fknowledge-distillation_ddsp-decoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgregogiudici%2Fknowledge-distillation_ddsp-decoder/lists"}