{"id":18417580,"url":"https://github.com/xiaomi/kaldi-onnx","last_synced_at":"2025-04-09T14:12:45.665Z","repository":{"id":40984169,"uuid":"186979319","full_name":"XiaoMi/kaldi-onnx","owner":"XiaoMi","description":"Kaldi model converter to ONNX","archived":false,"fork":false,"pushed_at":"2023-01-27T01:11:01.000Z","size":77,"stargazers_count":241,"open_issues_count":24,"forks_count":59,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-04-02T12:12:55.065Z","etag":null,"topics":["android","ios","kaldi","mace","onnx","speech-recognition"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/XiaoMi.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.txt","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-16T07:51:53.000Z","updated_at":"2025-03-10T18:55:59.000Z","dependencies_parsed_at":"2023-02-15T03:46:43.165Z","dependency_job_id":null,"html_url":"https://github.com/XiaoMi/kaldi-onnx","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiaoMi%2Fkaldi-onnx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiaoMi%2Fkaldi-onnx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiaoMi%2Fkaldi-onnx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/XiaoMi%2Fkaldi-onnx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/XiaoMi","download_url":"https://codeload.github.com/XiaoMi/kaldi-onnx/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248054193,"owners_count":21039952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["android","ios","kaldi","mace","onnx","speech-recognition"],"created_at":"2024-11-06T04:10:06.303Z","updated_at":"2025-04-09T14:12:45.639Z","avatar_url":"https://github.com/XiaoMi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kaldi-ONNX Converter\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![Build Status](https://api.travis-ci.org/XiaoMi/kaldi-onnx.svg?branch=master)](https://travis-ci.org/XiaoMi/kaldi-onnx)\n\n[English](README.md) | [中文](README_zh.md)\n\n**Kaldi-ONNX** is a tool for porting [Kaldi Speech Recognition Toolkit](https://github.com/kaldi-asr/kaldi) \nneural network models to [ONNX](https://github.com/onnx/onnx) models for inference.\nWith the converted ONNX model, you can use [MACE](https://github.com/XiaoMi/mace)\nto speedup the inference on **Android**, **iOS**, **Linux** or **Windows** devices with\nhighly optimized NEON kernels (more heterogeneous devices will be supported in the future).\n\nThis tool supports converting both Nnet2 and Nnet3 models. Almost all components\nin Nnet2 and Nnet3 models are supported, and the available components are listed\nin converter/common.py. \n\nBesides DNN, many speech recognition models are using RNN or TDNN networks.\nTo make them more portable and efficient, this tool will convert these RNN and\nTDNN networks to DAG-like networks.\n\n## Usage\n\n### 1. Dependencies\n\n```sh\npip install -r requirements.txt\n```\n\n### 2. Prepare models\nThis tool only supports Kaldi's text model as an input.\n\nIf you have a binary model, Kaldi's `nnet-am-copy` or `nnet3-copy` tool can help you get a text one:\n\nNnet2 Model\n\n```sh\npath/to/kaldi/src/nnet2bin/nnet-am-copy --binary=false          \\\n                                        final.raw text.mdl\n```\n\nNnet3 Model\n\n```sh\npath/to/kaldi/src/nnet3bin/nnet3-copy --binary=false          \\\n                                      --raw=true              \\\n                                      --prepare-for-test=true \\\n                                      final.raw text.mdl\n```\n\nDon't forget to use the `--prepare-for-test=true` and `--raw=true`option to optimize the model.\n\nBefore converting, you need to use `nnet-am-info` or `nnet3-am-info` to get left_context\\right_context\\modulus,\nwhich is required for converting.\n\nMore details about kaldi's tools  are in [Kaldi's Documentation](http://kaldi-asr.org/doc/).\n\n\n### 3. Convert\nOne command to convert:\n\n```sh\npython converter/convert.py --input=models/kaldi/model_name.mdl  \\\n                            --output=models/onnx/model_name.onnx \\\n                            --transition-model=path/to/save/transition_model.trans \\\n                            --conf=path/to/save/configuration.conf \\\n                            --chunk-size=20 \\\n                            --left-context=left_context \\\n                            --right-context=right_context \\\n                            --modulus=modulus \\\n                            --subsample-factor=subsample_factor \\\n                            --nnet-type=3\n```\n\n\n### 4. Graph review\nAfter converting, there is a graphic tool for you to review the onnx model: [ONNX Model Viewer](https://lutzroeder.github.io/netron/).\n\n\n### 5. Deployment\n\nTo run the ONNX models, an inference framework with Kaldi specific operators support is needed.\nCurrently, [MACE](https://github.com/XiaoMi/mace) has Kaldi-ONNX support which is primarily optimized for mobile phones and IoT devices.\n\nFor more details about deployment, please refer to [MACE documents](https://mace.readthedocs.io/en/latest/).\n\n\n### 6. Validation\n\nSince MACE already supports most frequently used components in Kaldi.\n We can use it to validate converted Kaldi models.\nThe validation process is giving the same inputs to Kaldi and MACE's computation and\n check if they will output the same results.\n\n**Generate random input**\n\nWe provide a tool to generate the same random input data for Kaldi and MACE's computation.\n\n```sh\npython tools/generate_inputs.py --input-dim=40 \\\n                                --chunk-size=20 \\\n                                --left-context=left_context \\\n                                --right-context=right_contex \\\n                                --kaldi-input_file=path/to/kaldi/input/test_input.ark \\\n                                --mace-input_file=path/to/mace/input/test_input.npy\n\n```\n'input_dim' and 'chunk_size' are model's input dim and chunk, can be specified by case.\n'kaldi_input_file' and 'mace_input_file' are paths for saving generated data files. \n\n**Compute in Kaldi**\n\nKaldi has command line tools for computing the model's propogation.\n\nNnet2:\n\n```sh\npath/to/kaldi/src/nnet2bin/nnet-am-compute  --chunk-size=chunk_size \\\n                                            path/to/kaldi/model/file/final.mdl \\\n                                            ark,t:/path/to/kaldi/input/file/test_input.ark \\\n                                            ark,t:path/to/save/output/data/test_output.ark\n\n```\n\nNnet3:\n\n```sh\npath/to/kaldi/src/nnet3-compute  --frames-per-chunk=chunk_size \\\n                                 path/to/kaldi/model/file/final.mdl \\\n                                 ark,t:/path/to/kaldi/input/file/test_input.ark \\\n                                 ark,t:path/to/save/output/data/test_output.ark\n\n```\n**Convert output data file**\n\nAfter running `nnet-compute`, we'll get Kaldi's output data in text format .\nBecause MACE supports numpy data file as input or output, we need to convert Kaldi's output data file to numpy format.\n\nThe script tools/kaldi_to_mace.py will help you doing this convertion.\n\n```sh\npython tools/kaldi_to_mace.py  --input=path/to/kaldi/data/file.ark \\\n                               --output=path/to/save/numpy/data/file.npy\n```\n\n**Prepare config file for MACE**\n\nHere is an example yaml config file for fisher english model.\n\n```yaml\n# fisher_english.yml\n\nlibrary_name: fisher_english_8\ntarget_abis: [armeabi-v7a, arm64-v8a]\nmodel_graph_format: file\nmodel_data_format: file\nmodels:\n  fisher_english_8:\n    platform: onnx\n    model_file_path: https://cnbj1.fds.api.xiaomi.com/mace/miai-models/onnx/kaldi/nnet2/fisher_english_8_nnet_a.onnx\n    model_sha256_checksum: e27d8147995b0a68e1367d060dc4f41c0f434043992a52548ff961e4e1e87e6c\n    subgraphs:\n      - input_tensors:\n          - 0\n        input_shapes:\n          - 1,20,140\n        output_tensors:\n          - 17\n        output_shapes:\n          - 1,20,7880\n        backend: kaldi\n        input_data_formats: NONE\n        output_data_formats: NONE\n        validation_inputs_data:\n            - https://cnbj1.fds.api.xiaomi.com/mace/miai-models/onnx/kaldi/data/kaldi_input_20_140.npy\n        validation_outputs_data:\n            - https://cnbj1.fds.api.xiaomi.com/mace/miai-models/onnx/kaldi/data/test_fisher_english_8_20_140_out.npy\n    backend: kaldi\n    runtime: cpu\n    limit_opencl_kernel_time: 0\n    nnlib_graph_mode: 0\n    obfuscate: 0\n\n```\n\n**Validate**\n\nConvert and validate the model in MACE:\n\n```sh\ncd path/to/mace\npython tools/converter.py convert --config=path/to/fisher_english.yml\npython tools/converter.py run --config=path/to/fisher_english.yml --validate\n\n```\nThe command will give you the similarity between MACE's and Kaldi's output results.\n\nMore details about how to use MACE, please step to [MACE documents](https://mace.readthedocs.io/en/latest/).\n\n### 5. Examples\n\nWe have converted numbers of Kaldi's Nnet2 and Nnet3 models, and put them in [MACE Model Zoo](https://github.com/XiaoMi/mace-models).\n\n\n## Communication\n* GitHub issues: bug reports, usage issues, feature requests\n\n\n## License\n[Apache License 2.0](LICENSE).\n\n\n## Contributing\n\nPython code should conform to [PEP8 Style Guide for Python Code](https://www.python.org/dev/peps/pep-0008/).\n\nYou can use [pycodestyle](https://github.com/PyCQA/pycodestyle) to check the style.\n\n```commandline\npycodestyle $(find . -name \"*.py\")\n```\n\nAny kind of contribution is welcome. For bug reports, feature requests,\nplease just open an issue without any hesitation. For code contributions, it's\nstrongly suggested to open an issue for discussion first.\n\n\n## Acknowledgement\nkaldi-onnx learned a lot from the following projects during the development:\n* [Kaldi Speech Recognition Toolkit](https://github.com/kaldi-asr/kaldi),\n* [ONNX](https://github.com/onnx/onnx).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaomi%2Fkaldi-onnx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxiaomi%2Fkaldi-onnx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxiaomi%2Fkaldi-onnx/lists"}