{"id":13396942,"url":"https://github.com/keunwoochoi/kapre","last_synced_at":"2025-05-15T09:08:20.098Z","repository":{"id":41465442,"uuid":"76486613","full_name":"keunwoochoi/kapre","owner":"keunwoochoi","description":"kapre: Keras Audio Preprocessors","archived":false,"fork":false,"pushed_at":"2023-10-23T02:52:41.000Z","size":5091,"stargazers_count":930,"open_issues_count":17,"forks_count":146,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-05-13T09:08:33.973Z","etag":null,"topics":["audio","kapre-layers","keras","keras-audio-preprocessors","melspectrogram","preprocess","shot","spectrogram","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/keunwoochoi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":null,"patreon":"keunwoochoi","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2016-12-14T18:36:36.000Z","updated_at":"2025-05-09T14:18:20.000Z","dependencies_parsed_at":"2024-01-08T12:17:48.456Z","dependency_job_id":null,"html_url":"https://github.com/keunwoochoi/kapre","commit_stats":{"total_commits":186,"total_committers":16,"mean_commits":11.625,"dds":"0.12365591397849462","last_synced_commit":"af4eb541d733cfed3ec57b10579307e9f5613359"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keunwoochoi%2Fkapre","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keunwoochoi%2Fkapre/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keunwoochoi%2Fkapre/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keunwoochoi%2Fkapre/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/keunwoochoi","download_url":"https://codeload.github.com/keunwoochoi/kapre/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254310520,"owners_count":22049470,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","kapre-layers","keras","keras-audio-preprocessors","melspectrogram","preprocess","shot","spectrogram","tensorflow"],"created_at":"2024-07-30T18:01:08.210Z","updated_at":"2025-05-15T09:08:15.090Z","avatar_url":"https://github.com/keunwoochoi.png","language":"Python","funding_links":["https://patreon.com/keunwoochoi"],"categories":["Audio","资源列表","音频","音频处理","Python","Audio [🔝](#readme)","Machine Learning Audio","Audio Related Packages"],"sub_categories":["音频","Drone Frames","IO router / deconstructed loops anchor"],"readme":"# Kapre\nKeras Audio Preprocessors - compute STFT, ISTFT, Melspectrogram, and others on GPU real-time.\n \nTested on Python 3.6 and 3.7\n\n## Why Kapre?\n\n### vs. Pre-computation\n\n* You can optimize DSP parameters\n* Your model deployment becomes much simpler and consistent.\n* Your code and model has less dependencies\n\n### vs. Your own implementation\n\n* Quick and easy!\n* Consistent with 1D/2D tensorflow batch shapes\n* Data format agnostic (`channels_first` and `channels_last`)\n* Less error prone - Kapre layers are tested against Librosa (stft, decibel, etc) - which is (trust me) *trickier* than you think.\n* Kapre layers have some extended APIs from the default `tf.signals` implementation such as..\n  - A perfectly invertible `STFT` and `InverseSTFT` pair\n  - Mel-spectrogram with more options\n* Reproducibility - Kapre is available on pip with versioning   \n\n## Workflow with Kapre\n\n1. Preprocess your audio dataset. Resample the audio to the right sampling rate and store the audio signals (waveforms).\n2. In your ML model, add Kapre layer e.g. `kapre.time_frequency.STFT()` as the first layer of the model.\n3. The data loader simply loads audio signals and feed them into the model\n4. In your hyperparameter search, include DSP parameters like `n_fft` to boost the performance.\n5. When deploying the final model, all you need to remember is the sampling rate of the signal. No dependency or preprocessing!\n\n## Installation\n \n```sh\npip install kapre\n```\n\n## API Documentation\n\nPlease refer to Kapre API Documentation at https://kapre.readthedocs.io\n\n## One-shot example\n\n```python\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, BatchNormalization, ReLU, GlobalAveragePooling2D, Dense, Softmax\nfrom kapre import STFT, Magnitude, MagnitudeToDecibel\nfrom kapre.composed import get_melspectrogram_layer, get_log_frequency_spectrogram_layer\n\n# 6 channels (!), maybe 1-sec audio signal, for an example.\ninput_shape = (44100, 6)\nsr = 44100\nmodel = Sequential()\n# A STFT layer\nmodel.add(STFT(n_fft=2048, win_length=2018, hop_length=1024,\n               window_name=None, pad_end=False,\n               input_data_format='channels_last', output_data_format='channels_last',\n               input_shape=input_shape))\nmodel.add(Magnitude())\nmodel.add(MagnitudeToDecibel())  # these three layers can be replaced with get_stft_magnitude_layer()\n# Alternatively, you may want to use a melspectrogram layer\n# melgram_layer = get_melspectrogram_layer()\n# or log-frequency layer\n# log_stft_layer = get_log_frequency_spectrogram_layer() \n\n# add more layers as you want\nmodel.add(Conv2D(32, (3, 3), strides=(2, 2)))\nmodel.add(BatchNormalization())\nmodel.add(ReLU())\nmodel.add(GlobalAveragePooling2D())\nmodel.add(Dense(10))\nmodel.add(Softmax())\n\n# Compile the model\nmodel.compile('adam', 'categorical_crossentropy') # if single-label classification\n\n# train it with raw audio sample inputs\n# for example, you may have functions that load your data as below.\nx = load_x() # e.g., x.shape = (10000, 6, 44100)\ny = load_y() # e.g., y.shape = (10000, 10) if it's 10-class classification\n# then..\nmodel.fit(x, y)\n# Done!\n```\n\n* See the Jupyter notebook at the [example folder](https://github.com/keunwoochoi/kapre/tree/master/examples)\n\n## Tflite compatbility\n\nThe `STFT` layer is not tflite compatible (due to `tf.signal.stft`). To create a tflite\ncompatible model, first train using the normal `kapre` layers then create a new\nmodel replacing `STFT` and `Magnitude` with `STFTTflite`, `MagnitudeTflite`.\nTflite compatible layers are restricted to a batch size of 1 which prevents use\nof them during training.\n\n```python\n# assumes you have run the one-shot example above.\nfrom kapre import STFTTflite, MagnitudeTflite\nmodel_tflite = Sequential()\n\nmodel_tflite.add(STFTTflite(n_fft=2048, win_length=2018, hop_length=1024,\n               window_name=None, pad_end=False,\n               input_data_format='channels_last', output_data_format='channels_last',\n               input_shape=input_shape))\nmodel_tflite.add(MagnitudeTflite())\nmodel_tflite.add(MagnitudeToDecibel())  \nmodel_tflite.add(Conv2D(32, (3, 3), strides=(2, 2)))\nmodel_tflite.add(BatchNormalization())\nmodel_tflite.add(ReLU())\nmodel_tflite.add(GlobalAveragePooling2D())\nmodel_tflite.add(Dense(10))\nmodel_tflite.add(Softmax())\n\n# load the trained weights into the tflite compatible model.\nmodel_tflite.set_weights(model.get_weights())\n```\n\n# Citation\n\nPlease cite this paper if you use Kapre for your work.\n\n```\n@inproceedings{choi2017kapre,\n  title={Kapre: On-GPU Audio Preprocessing Layers for a Quick Implementation of Deep Neural Network Models with Keras},\n  author={Choi, Keunwoo and Joo, Deokjin and Kim, Juho},\n  booktitle={Machine Learning for Music Discovery Workshop at 34th International Conference on Machine Learning},\n  year={2017},\n  organization={ICML}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeunwoochoi%2Fkapre","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkeunwoochoi%2Fkapre","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeunwoochoi%2Fkapre/lists"}