{"id":15631420,"url":"https://github.com/csukuangfj/kaldifeat","last_synced_at":"2025-04-08T11:09:45.184Z","repository":{"id":37453793,"uuid":"342163168","full_name":"csukuangfj/kaldifeat","owner":"csukuangfj","description":"Kaldi-compatible online \u0026 offline feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and  autograd - Provide C++ \u0026 Python API","archived":false,"fork":false,"pushed_at":"2025-03-07T06:23:14.000Z","size":10709,"stargazers_count":197,"open_issues_count":4,"forks_count":37,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-01T10:06:59.566Z","etag":null,"topics":["cpp","fbank","features-extraction","kaldi","mfcc","online-feature-extractor","plp","python","pytorch","streaming-feature-extractor"],"latest_commit_sha":null,"homepage":"https://csukuangfj.github.io/kaldifeat","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csukuangfj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-25T07:45:43.000Z","updated_at":"2025-03-28T08:12:05.000Z","dependencies_parsed_at":"2023-02-05T11:02:24.428Z","dependency_job_id":"c99e03a0-245b-45d8-be8e-e77244168dde","html_url":"https://github.com/csukuangfj/kaldifeat","commit_stats":{"total_commits":113,"total_committers":4,"mean_commits":28.25,"dds":"0.026548672566371723","last_synced_commit":"bf2047023fd7c551c94673b8ead61a929e1243df"},"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csukuangfj%2Fkaldifeat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csukuangfj%2Fkaldifeat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csukuangfj%2Fkaldifeat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csukuangfj%2Fkaldifeat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csukuangfj","download_url":"https://codeload.github.com/csukuangfj/kaldifeat/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247829511,"owners_count":21002997,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","fbank","features-extraction","kaldi","mfcc","online-feature-extractor","plp","python","pytorch","streaming-feature-extractor"],"created_at":"2024-10-03T10:40:22.459Z","updated_at":"2025-04-08T11:09:45.165Z","avatar_url":"https://github.com/csukuangfj.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kaldifeat\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"/doc/source/images/os-green.svg\"\u003e\n\u003cimg src=\"/doc/source/images/python_ge_3.6-blue.svg\"\u003e\n\u003cimg src=\"/doc/source/images/pytorch_ge_1.5.0-green.svg\"\u003e\n\u003cimg src=\"/doc/source/images/cuda_ge_10.1-orange.svg\"\u003e\n\u003c/div\u003e\n\n[![Documentation Status](https://github.com/csukuangfj/kaldifeat/actions/workflows/build-doc.yml/badge.svg)](https://csukuangfj.github.io/kaldifeat/)\n\n**Documentation**: \u003chttps://csukuangfj.github.io/kaldifeat\u003e\n\n**Note**: If you are looking for a version that does not depend on PyTorch,\nplease see \u003chttps://github.com/csukuangfj/kaldi-native-fbank\u003e\n\n# Installation\n\nRefer to\n\u003chttps://csukuangfj.github.io/kaldifeat/installation/from_wheels.html\u003e\nfor installation.\n\n\u003e Never use `pip install kaldifeat`\n\n\u003e Never use `pip install kaldifeat`\n\n\u003e Never use `pip install kaldifeat`\n\n\n\n\u003csub\u003e\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003eComments\u003c/th\u003e\n\u003cth\u003eOptions\u003c/th\u003e\n\u003cth\u003eFeature Computer\u003c/th\u003e\n\u003cth\u003eUsage\u003c/th\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eFbank for \u003ca href=\"https://github.com/openai/whisper\"\u003eWhisper\u003c/a\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.WhisperFbankOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.WhisperFbank\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.WhisperFbankOptions()\nopts.device = torch.device('cuda', 0)\nfbank = kaldifeat.WhisperFbank(opts)\nfeatures = fbank(wave)\n\u003c/pre\u003e\nSee \u003ca href=\"https://github.com/csukuangfj/kaldifeat/pull/82\"\u003e#82\u003c/a\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eFbank for \u003ca href=\"https://github.com/openai/whisper\"\u003eWhisper-V3\u003c/a\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.WhisperFbankOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.WhisperFbank\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.WhisperFbankOptions()\nopts.num_mels = 128\nopts.device = torch.device('cuda', 0)\nfbank = kaldifeat.WhisperFbank(opts)\nfeatures = fbank(wave)\n\u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eFBANK\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.FbankOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.Fbank\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.FbankOptions()\nopts.device = torch.device('cuda', 0)\nopts.frame_opts.window_type = 'povey'\nfbank = kaldifeat.Fbank(opts)\nfeatures = fbank(wave)\n\u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eStreaming FBANK\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.FbankOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.OnlineFbank\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\nSee \u003ca href=\"./kaldifeat/python/tests/test_fbank.py\"\u003e\n./kaldifeat/python/tests/test_fbank.py\n\u003c/a\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eMFCC\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.MfccOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.Mfcc\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.MfccOptions();\nopts.num_ceps = 13\nmfcc = kaldifeat.Mfcc(opts)\nfeatures = mfcc(wave)\n\u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eStreaming MFCC\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.MfccOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.OnlineMfcc\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\nSee \u003ca href=\"./kaldifeat/python/tests/test_mfcc.py\"\u003e\n./kaldifeat/python/tests/test_mfcc.py\n\u003c/a\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003ePLP\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.PlpOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.Plp\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.PlpOptions();\nopts.mel_opts.num_bins = 23\nplp = kaldifeat.Plp(opts)\nfeatures = plp(wave)\n\u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eStreaming PLP\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.PlpOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.OnlinePlp\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\nSee \u003ca href=\"./kaldifeat/python/tests/test_plp.py\"\u003e\n./kaldifeat/python/tests/test_plp.py\n\u003c/a\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003eSpectorgram\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.SpectrogramOptions\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ekaldifeat.Spectrogram\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\n\u003cpre lang=\"python\"\u003e\nopts = kaldifeat.SpectrogramOptions();\nprint(opts)\nspectrogram = kaldifeat.Spectrogram(opts)\nfeatures = spectrogram(wave)\n\u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003c/sub\u003e\n\n\nFeature extraction compatible with `Kaldi` using PyTorch, supporting\nCUDA, batch processing, chunk processing, and autograd.\n\nThe following kaldi-compatible commandline tools are implemented:\n\n  - `compute-fbank-feats`\n  - `compute-mfcc-feats`\n  - `compute-plp-feats`\n  - `compute-spectrogram-feats`\n\n(**NOTE**: We will implement other types of features, e.g., Pitch, ivector, etc, soon.)\n\n**HINT**: It supports also streaming feature extractors for Fbank, MFCC, and Plp.\n\n# Usage\n\nLet us first generate a test wave using sox:\n\n```bash\n# generate a wave of 1.2 seconds, containing a sine-wave\n# swept from 300 Hz to 3300 Hz\nsox -n -r 16000 -b 16 test.wav synth 1.2 sine 300-3300\n```\n\n**HINT**: Download [test.wav][test_wav].\n\n[test_wav]: kaldifeat/python/tests/test_data/test.wav\n\n## Fbank\n\n```python\nimport torchaudio\n\nimport kaldifeat\n\nfilename = \"./test.wav\"\nwave, samp_freq = torchaudio.load(filename)\n\nwave = wave.squeeze()\n\nopts = kaldifeat.FbankOptions()\nopts.frame_opts.dither = 0\n# Yes, it has same options like `Kaldi`\n\nfbank = kaldifeat.Fbank(opts)\nfeatures = fbank(wave)\n```\n\nTo compute features that are compatible with `Kaldi`, wave samples have to be\nscaled to the range `[-32768, 32768]`. **WARNING**: You don't have to do this if\nyou don't care about the compatibility with `Kaldi`.\n\nThe following is an example:\n\n```python\nwave *= 32768\nfbank = kaldifeat.Fbank(opts)\nfeatures = fbank(wave)\nprint(features[:3])\n```\n\nThe output is:\n\n```\ntensor([[15.0074, 21.1730, 25.5286, 24.4644, 16.6994, 13.8480, 11.2087, 11.7952,\n         10.3911, 10.4491, 10.3012,  9.8743,  9.6997,  9.3751,  9.3476,  9.3559,\n          9.1074,  9.0032,  9.0312,  8.8399,  9.0822,  8.7442,  8.4023],\n        [13.8785, 20.5647, 25.4956, 24.6966, 16.9541, 13.9163, 11.3364, 11.8449,\n         10.2565, 10.5871, 10.3484,  9.7474,  9.6123,  9.3964,  9.0695,  9.1177,\n          8.9136,  8.8425,  8.5920,  8.8315,  8.6226,  8.8605,  8.9763],\n        [13.9475, 19.9410, 25.4494, 24.9051, 17.0004, 13.9207, 11.6667, 11.8217,\n         10.3411, 10.7258, 10.0983,  9.8109,  9.6762,  9.4218,  9.1246,  8.7744,\n          9.0863,  8.7488,  8.4695,  8.6710,  8.7728,  8.7405,  8.9824]])\n```\n\nYou can compute the fbank feature for the same wave with `Kaldi` using the following commands:\n\n```bash\necho \"1 test.wav\" \u003e test.scp\ncompute-fbank-feats --dither=0 scp:test.scp ark,t:test.txt\nhead -n4 test.txt\n```\n\nThe output is:\n\n```\n1  [\n  15.00744 21.17303 25.52861 24.46438 16.69938 13.84804 11.2087 11.79517 10.3911 10.44909 10.30123 9.874329 9.699727 9.37509 9.347578 9.355928 9.107419 9.00323 9.031268 8.839916 9.082197 8.744139 8.40221\n  13.87853 20.56466 25.49562 24.69662 16.9541 13.91633 11.33638 11.84495 10.25656 10.58718 10.34841 9.747416 9.612316 9.39642 9.06955 9.117751 8.913527 8.842571 8.59212 8.831518 8.622513 8.86048 8.976251\n  13.94753 19.94101 25.4494 24.90511 17.00044 13.92074 11.66673 11.82172 10.34108 10.72575 10.09829 9.810879 9.676199 9.421767 9.124647 8.774353 9.086291 8.74897 8.469534 8.670973 8.772754 8.740549 8.982433\n```\n\nYou can see that ``kaldifeat`` produces the same output as `Kaldi` (within some tolerance due to numerical precision).\n\n\n**HINT**: Download [test.scp][test_scp] and [test.txt][test_txt].\n\n[test_scp]: kaldifeat/python/tests/test_data/test.scp\n[test_txt]: kaldifeat/python/tests/test_data/test.txt\n\n\nTo use GPU, you can use:\n\n```python\nimport torch\n\nopts = kaldifeat.FbankOptions()\nopts.device = torch.device(\"cuda\", 0)\n\nfbank = kaldifeat.Fbank(opts)\nfeatures = fbank(wave.to(opts.device))\n```\n\n## MFCC, PLP, Spectrogram\n\nTo compute MFCC features, please replace `kaldifeat.FbankOptions` and `kaldifeat.Fbank`\nwith `kaldifeat.MfccOptions` and `kaldifeat.Mfcc`, respectively. The same goes\nfor `PLP` and `Spectrogram`.\n\nPlease refer to\n\n  - [kaldifeat/python/tests/test_fbank.py](kaldifeat/python/tests/test_fbank.py)\n  - [kaldifeat/python/tests/test_mfcc.py](kaldifeat/python/tests/test_mfcc.py)\n  - [kaldifeat/python/tests/test_plp.py](kaldifeat/python/tests/test_plp.py)\n  - [kaldifeat/python/tests/test_spectrogram.py](kaldifeat/python/tests/test_spectrogram.py)\n  - [kaldifeat/python/tests/test_frame_extraction_options.py](kaldifeat/python/tests/test_frame_extraction_options.py)\n  - [kaldifeat/python/tests/test_mel_bank_options.py](kaldifeat/python/tests/test_mel_bank_options.py)\n  - [kaldifeat/python/tests/test_fbank_options.py](kaldifeat/python/tests/test_fbank_options.py)\n  - [kaldifeat/python/tests/test_mfcc_options.py](kaldifeat/python/tests/test_mfcc_options.py)\n  - [kaldifeat/python/tests/test_spectrogram_options.py](kaldifeat/python/tests/test_spectrogram_options.py)\n  - [kaldifeat/python/tests/test_plp_options.py](kaldifeat/python/tests/test_plp_options.py)\n\nfor more examples.\n\n**HINT**: In the examples, you can find that\n\n- ``kaldifeat`` supports batch processing as well as chunk processing\n- ``kaldifeat`` uses the same options as `Kaldi`'s `compute-fbank-feats` and `compute-mfcc-feats`\n\n# Usage in other projects\n\n## icefall\n\n[icefall](https://github.com/k2-fsa/icefall) uses kaldifeat to extract features for a pre-trained model.\n\nSee \u003chttps://github.com/k2-fsa/icefall/blob/master/egs/librispeech/ASR/conformer_ctc/pretrained.py\u003e.\n\n## k2\n\n[k2](https://github.com/k2-fsa/k2) uses kaldifeat's C++ API.\n\nSee \u003chttps://github.com/k2-fsa/k2/blob/v2.0-pre/k2/torch/csrc/features.cu\u003e.\n\n## lhotse\n\n[lhotse](https://github.com/lhotse-speech/lhotse) uses kaldifeat to extract features on GPU.\n\nSee \u003chttps://github.com/lhotse-speech/lhotse/blob/master/lhotse/features/kaldifeat.py\u003e.\n\n## sherpa\n\n[sherpa](https://github.com/k2-fsa/sherpa) uses kaldifeat for streaming feature\nextraction.\n\nSee \u003chttps://github.com/k2-fsa/sherpa/blob/master/sherpa/bin/pruned_stateless_emformer_rnnt2/decode.py\u003e\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsukuangfj%2Fkaldifeat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsukuangfj%2Fkaldifeat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsukuangfj%2Fkaldifeat/lists"}