{"id":15047867,"url":"https://github.com/narasimha1997/wavenet-stt","last_synced_at":"2025-07-05T07:03:38.198Z","repository":{"id":104768412,"uuid":"295449868","full_name":"Narasimha1997/wavenet-stt","owner":"Narasimha1997","description":"An end-to-end speech recognition system with Wavenet. Built using C++ and python.","archived":false,"fork":false,"pushed_at":"2020-09-21T04:15:13.000Z","size":15315,"stargazers_count":21,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-04T00:19:05.628Z","etag":null,"topics":["cplusplus-11","pybind","python","python3","speech-recognition","tensorflow","tensorflow2","tensorflowlite","wavenet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Narasimha1997.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-14T14:56:41.000Z","updated_at":"2023-01-18T12:05:18.000Z","dependencies_parsed_at":"2023-05-29T17:45:30.054Z","dependency_job_id":null,"html_url":"https://github.com/Narasimha1997/wavenet-stt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Narasimha1997/wavenet-stt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fwavenet-stt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fwavenet-stt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fwavenet-stt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fwavenet-stt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Narasimha1997","download_url":"https://codeload.github.com/Narasimha1997/wavenet-stt/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Narasimha1997%2Fwavenet-stt/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263699717,"owners_count":23497958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cplusplus-11","pybind","python","python3","speech-recognition","tensorflow","tensorflow2","tensorflowlite","wavenet"],"created_at":"2024-09-24T21:05:23.722Z","updated_at":"2025-07-05T07:03:38.172Z","avatar_url":"https://github.com/Narasimha1997.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### STT-Wavenet\nPython and C++ implementation of end-to-end sentence level Speech Recognition using DeepMind's recent research on audio processing and synthesis. This is based on [WaveNet: A Generative Model for Raw Audio](https://arxiv.org/pdf/1609.03499.pdf) where DeepMind proposed a neural network architecture that could generate human-like audio from text, the model is also capable of performing speech-to-text. This repo provides speech-to-text implementation of Wavenet. The model takes Mel-spectograph as input and produces text as output using wavenet + beam search decoder.\n\n![Wavenet STT](./images/architecture.png)\n\n#### Modifying the architecture/making changes to the exporter:\nThose who wish to modify or play with wavenet architecture can go to `core` directory.\nRefer [README.md](./core/README.md)\n\n#### Building C++ api (libtensorflow_cc)\nTO build C++ api, you have to build tensorflow from scratch along with its dependencies as a monolithic shared library, also make sure the headers are properly exported. If you don't want to build tensorflow, use pre-built shared libraries from [FloopCZ/tensorflow_cc](https://github.com/FloopCZ/tensorflow_cc). I would recommend building tensorflow from scratch as it properly compiles to your hardware, using prebuilt shared libraries can lead to segmentation faults and illegal instruction execution attempts as they would have compiled tensorflow with different versions of gcc and different hardware optminzations that your processor lacks.\n\n##### Building tensorflow from scratch:\n1. Install bazel and clone tensorflow repository, run `config` script and answer the questions carefully.\n2. Build `libtensorflow_cc`:\n    ```\n    bazel build -c opt --config monolithic //tensorflow:libtensorflow_cc.so\n    ```\n3. Export headers:\n    ```\n    bazel build -c opt --config monolithic //tensorflow:install_headers\n    ```\n##### Build the wavenet module\n1. Install `pybind11` and python headers.\n    ```\n    sudo apt install python3-dev \u0026\u0026\n    pip3 install pybind11\n    ```\n2. Go to `platform/buld_env`\n3. Make sure you properly set these env variables:\n    ```shell\n    TENSORFLOW_CC_LIBS_PATH=${HOME}/Documents/installation/tensorflow/lib        #libtensorflow_cc.so* directory\n    TENSORFLOW_CC_INCLUDE_PATH=${HOME}/Documents/installation/tensorflow/include #tensorflow headers directory\n\n    #misc : keep them default\n    present_dir=$(pwd)\n    SST_SOURCE_DIR=${present_dir}/../cc/src \n    SST_INCLUDE_PATH=${present_dir}/../cc/include\n    SST_PYTHON_PATH=${present_dir}/../wavenetsst/wavenetpy\n    ```\n4. Build wavenet CPython module\n    ```\n    ./tensorflow_env.sh python\n    ```\n\n\n##### Using python wavenet module \nThe wavenet python module wavenetpy is located at `platform/wavenetstt`, the module requires wavenet CPython shared library. Since static build is yet to be implemeted, the shared library dynamically links with `libtensorflow_cc` during runtime. So, make sure you export a proper `LD_LIBRARY_PATH`.\n\n```shell\nTENSORFLOW_CC_LIBS_PATH=${HOME}/Documents/installation/tensorflow/lib\nLD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${TENSORFLOW_CC_LIBS_PATH}\n```\n\nInstall requirements -\u003e numpy and librosa\n```\npip3 install -r requirements.txt\n```\n\nExample : Running speech recognition\n```python3\nfrom wavenetpy import WavenetSTT\n#load the model\nwavenet = WavenetSTT('../../pb/wavenet-stt.pb')\n\n#pass the audio file\nresult = wavenet.infer_on_file('test.wav')\nprint(result)\n```\n\n##### TODO and Roadmap:\n1. Build static library to avoid dynamic linking with `libtensorflow_cc`\n2. We are using `librosa` for MFCC, the goal is to use custom C++ implementation.\n3. Use custom C++ `ctc_beam_search_decoder` because it is not supported in tensorflow lite.\n4. Provide a Dockerfile\n5. Implement Tensorflow Lite implementation for embedded devices and android.\n6. Add Tensorflow.js support.\n7. Optimize C++ code.\n8. Provide CI/CD pipeline for C++ build. \n9. Provide a way to directly access mfcc pointer instead of memory copy. This is not possible as now because of the limitation in Tensorflow C++ api. In other words, add a custom memory allocator for tensors.\n\n##### Contributor guide\nWe welcome contributors especially beginners. Contributors can :\n1. Raise issues\n2. Suggest features\n3. Fix issues and bugs\n4. Impelement features specified in TODO and Roadmap.\n\n##### Acknowledgements\n1. Deepmind\n2. [buriburisuri](https://github.com/buriburisuri/speech-to-text-wavenet) for providing the pretrained ckpt files and `wavenet.py`.\n3. [kingscraft](https://github.com/kingstarcraft/speech-to-text-wavenet2) for tensorflow reference implementation.\n3. Stackoverflow and Tensorflow Docs\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarasimha1997%2Fwavenet-stt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnarasimha1997%2Fwavenet-stt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnarasimha1997%2Fwavenet-stt/lists"}