{"id":15649113,"url":"https://github.com/dansuh17/jdcnet-pytorch","last_synced_at":"2025-04-30T15:23:01.572Z","repository":{"id":40954289,"uuid":"213160839","full_name":"dansuh17/jdcnet-pytorch","owner":"dansuh17","description":"pytorch implementation of JDCNet, singing voice detection and classification network","archived":false,"fork":false,"pushed_at":"2023-02-15T22:55:35.000Z","size":19928,"stargazers_count":50,"open_issues_count":5,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-30T15:22:50.453Z","etag":null,"topics":["bilstm","deep-learning","lstm","melody","mir","music-information-retrieval","pytorch","singing-voice"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dansuh17.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-10-06T12:02:52.000Z","updated_at":"2025-01-16T18:04:36.000Z","dependencies_parsed_at":"2023-01-21T16:04:03.308Z","dependency_job_id":"8a186898-970b-4ddd-9246-90cee905e8cb","html_url":"https://github.com/dansuh17/jdcnet-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dansuh17%2Fjdcnet-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dansuh17%2Fjdcnet-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dansuh17%2Fjdcnet-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dansuh17%2Fjdcnet-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dansuh17","download_url":"https://codeload.github.com/dansuh17/jdcnet-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251729811,"owners_count":21634294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bilstm","deep-learning","lstm","melody","mir","music-information-retrieval","pytorch","singing-voice"],"created_at":"2024-10-03T12:27:59.686Z","updated_at":"2025-04-30T15:23:01.532Z","avatar_url":"https://github.com/dansuh17.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JDCNet-pytorch\n\nThis is a [PyTorch](https://pytorch.org/) re-implementation of \n_Kum et al. - \"Joint Detection and Classification of \nSinging Voice Melody Using Convolutional Recurrent Neural Networks\" (2019)_.\nThe proposed neural network model will be called **JDCNet** for convenience.\n\n- **paper**: [PDF](https://www.mdpi.com/2076-3417/9/7/1324)\n- **original Keras implementation**: [melodyExtraction_JDC](https://github.com/keums/melodyExtraction_JDC)\n\nThis is an attempt of implementing JDCNet as close as possible with the original paper.\nAny ambiguities in implementation details have been filled in by my own decisions,\nwhich account for any differences with the original author's implementation details.\n\n## Prerequisites\n\nMajor dependencies for this project are:\n\n- python \u003e= 3.6\n- pytorch \u003e= 1.2\n- librosa \u003e= 0.7.0\n- pytorch-land == 0.1.6 (train only)\n\nAny other required libraries are written in `requirements.txt`.\n\nThis project also uses a mini library called [pytorch-land](https://github.com/dansuh17/pytorch-land) \ncreated by myself that implements a general `Trainer` for pytorch based models.\nIt provides easy logging, native tensorboard support, \nand performs basic \"train-validate-test\" training sequence.\n\n[librosa](https://librosa.github.io/librosa/) is used for reading audio files.\n\n# JDCNet\n\nJDCNet is a singing voice melody detection and classification network.\nIt detects detection of whether there exists a noticeable singing voice in a certain frame, \nand, if exists, classifies the pitch of the sung note.\n\nThe pitch classification is done using a convolutional network with a bidirectional LSTM (BiLSTM) module attached at the end.\nIntermediate features for pitch classifier are utilized by the auxiliary detector network, \nalso a BiLSTM module, to aid the determination of voice existence.\n\nThe input is a log-magnitude spectrogram chunk that consists of 31 frames and 513 frequency bins.\n\nThe model predicts whether or not the voice exists for each frame, giving a `(31 x 2)` tensor output,\nand classifies the pitch into one of 722 classes that represent 721 different frequencies \nevenly distributed (in log scale) from notes D3 (MIDI=38) to B5 (MIDI=83) inclusive,\nand an extra 'non-voice' class.\n\n![jdcnet_architecture](assets/jdcnet_diagram.png)\n\n# Data Preprocess\n\n[MedleyDB's Melody Subset](https://zenodo.org/record/2628782#.XcvOPpIzZ24) \ndataset is used to train this model.\nAcquire the dataset, extract the contents, and run the preprocessing script \nto be ready for training.\n\n```shell\n./medleydb_preprocess.py --in_root \u003cmedleydb_root\u003e --out_root \u003coutput_root\u003e --metadata_path \u003cpath\u003e/\u003cto\u003e/\u003cmetadata_file\u003e.json\n```\n\n# Train\n\nYou must provide a configuration file to train the network. \nDefault configuration file with default parameters are provided as `default_config.json`.\nIn order to start training, run the script `train.py`.\n\n```shell\n./train.py --config default_config.json\n```\n\n# Singing voice melody extraction\n\nYou can generate a MIDI file containing extracted singing voice melody using the provided [pretrained model](/example_model).\n\n```shell\n./extract_melody.py --model example_model/jdcnet_model.pth --input_audio \u003cyour_audio\u003e.wav\n```\n\n# Generated Melody Audio Examples\n\nSome audible examples have been posted in [this post](https://dansuh17.github.io/2019/11/19/jdcnet.html), \nand example MIDI files are in the ['melody_results'](/melody_results) directory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdansuh17%2Fjdcnet-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdansuh17%2Fjdcnet-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdansuh17%2Fjdcnet-pytorch/lists"}