{"id":15047451,"url":"https://github.com/psavarmattas/speechtotext","last_synced_at":"2026-04-02T02:37:14.217Z","repository":{"id":164754499,"uuid":"408156329","full_name":"psavarmattas/SpeechToText","owner":"psavarmattas","description":"we shall build a very simple speech recognition system that takes our voice as input and produces the corresponding text by hearing the input.","archived":false,"fork":false,"pushed_at":"2024-02-03T20:01:34.000Z","size":972,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-13T23:13:43.722Z","etag":null,"topics":["facebook-api","ipython","librosa","machine-learning","numpy","python","pytorch","soundfile","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/psavarmattas.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.MD","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-19T14:57:51.000Z","updated_at":"2024-02-03T19:57:17.000Z","dependencies_parsed_at":"2024-02-03T20:47:53.741Z","dependency_job_id":null,"html_url":"https://github.com/psavarmattas/SpeechToText","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psavarmattas%2FSpeechToText","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psavarmattas%2FSpeechToText/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psavarmattas%2FSpeechToText/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/psavarmattas%2FSpeechToText/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/psavarmattas","download_url":"https://codeload.github.com/psavarmattas/SpeechToText/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243495495,"owners_count":20299923,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["facebook-api","ipython","librosa","machine-learning","numpy","python","pytorch","soundfile","transformers"],"created_at":"2024-09-24T20:58:43.044Z","updated_at":"2025-12-29T12:19:31.325Z","avatar_url":"https://github.com/psavarmattas.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Introduction\n\nYou may be frequently using Google Assistant or Apple’s Siri or even Amazon Alexa to find out quick answers on the web or to simply command something. These AI assistants are well known for understanding our speech commands and performing the desired tasks. They quickly respond to the speech commands like “OK, Google. What is the capital of Iceland?” or “Hey, Siri. Show me the recipe of Lasagna.” and display the accurate result. Have you ever wondered how they really do it? Today, we shall build a very simple speech recognition system that takes our voice as input and produces the corresponding text by hearing the input.\n\n![assistant image](https://github.com/psavarmattas/SpeechToText/blob/6f04d775b0bebbceec105a9930788feeaeb5c283/assets/image1.jpg)\n\n## TLDR Show me the code!\n\nIf you are impatient like me, this is practically the full source code that can be quickly copied, pasted, and executed through a Python file. Make sure to have a file named ‘my-audio.wav’ as your speech input. Also, make sure you have all the libraries installed.\n\nFor that you can install all the libraries one by one or you can run the command below:\n\n\u003cpre\u003e\u003ccode\u003e\npip install -r requirements.txt\n\u003c/code\u003e\u003c/pre\u003e\n\nIn the later part of the tutorial, we will be discussing what each of the lines is doing.\n\nHere’s the code!\n\n\u003cpre\u003e\u003ccode\u003e\nimport torch\nimport librosa\nimport numpy as np\nimport soundfile as sf\nfrom scipy.io import wavfile\nfrom IPython.display import Audio\nfrom transformers import Wav2Vec2ForCTC, Wav2Vec2Tokenizer\n\u003c/code\u003e\u003c/pre\u003e\n\n\u003cpre\u003e\u003ccode\u003e\ntokenizer = Wav2Vec2Tokenizer.from_pretrained(\"facebook/wav2vec2-base-960h\")\nmodel = Wav2Vec2ForCTC.from_pretrained(\"facebook/wav2vec2-base-960h\")\n\u003c/code\u003e\u003c/pre\u003e\n\n\u003cpre\u003e\u003ccode\u003e\nfile_name = 'my-audio.wav'\n\u003c/code\u003e\u003c/pre\u003e\n\n\u003cpre\u003e\u003ccode\u003e\ndata = wavfile.read(file_name)\nframerate = data[0]\nsounddata = data[1]\ntime = np.arange(0,len(sounddata))/framerate\ninput_audio, _ = librosa.load(file_name, sr=16000)\ninput_values = tokenizer(input_audio, return_tensors=\"pt\").input_values\nlogits = model(input_values).logits\npredicted_ids = torch.argmax(logits, dim=-1)\ntranscription = tokenizer.batch_decode(predicted_ids)[0]\nprint(transcription)\n\u003c/code\u003e\u003c/pre\u003e\n\n## Before we begin\nMake sure to check the full source code of this tutorial in [this Github repo.](https://github.com/psavarmattas/SpeechToText)\n\n## Wav2Vec: A Revolutionary Model\n\n![Wav2Vec Model](https://github.com/psavarmattas/SpeechToText/blob/2af3cdb7b90b46f4ab7a9f148b61fcf34a5d9ac6/assets/image2.png)\n\nWe will be using [Wave2Vec](https://ai.facebook.com/blog/wav2vec-20-learning-the-structure-of-speech-from-raw-audio/) — a state-of-the-art speech recognition approach by Facebook.\n\nThe researchers at Facebook describe this approach as:\n\nThere are thousands of languages spoken around the world, many with several different dialects, which presents a huge challenge for building high-quality speech recognition technology. It’s simply not feasible to obtain resources for each dialect and every language across the many possible domains (read speech, telephone speech, etc.). Our new model, wav2vec 2.0 , uses self-supervision to push the boundaries by learning from unlabeled training data to enable speech recognition systems for many more languages, dialects, and domains. With just one hour of labeled training data, wav2vec 2.0 outperforms the previous state of the art on the 100-hour subset of the LibriSpeech benchmark — using 100 times less labeled data.\n\nThere you go! Wav2Vec is a self-supervised model that aims to create a speech recognition system for several languages and dialects. With very little training data (roughly 100 times less labelled), the model has been able to outperform the previous state-of-the-art benchmark.\n\nThis model, like BERT, is trained by predicting speech units for masked audio segments. Speech audio, on the other hand, is a continuous signal that captures many features of the recording without being clearly segmented into words or other units. Wav2vec 2.0 addresses this problem by learning basic units of 25ms in order to learn high-level contextualized representations. These units are then utilized to characterize a wide range of spoken audio recordings, enhancing the robustness of wav2vec. With 100x less labelled training data, we can create voice recognition algorithms that surpass the best-semisupervised approaches. Thanks to self-supervised learning, Wav2vec 2.0 is part of machine learning models that rely less on labelled input. Self-supervision has aided in the advancement of image classification, video comprehension, and content comprehension systems. The algorithm could lead to advancements in speech technology for a wide range of languages, dialects, and domains, as well as for current systems.\n\n## Choosing our environment and libraries\n\nWe will be using [PyTorch](https://pytorch.org/), an open-source machine learning framework for this operation. Similarly, we will use [Transformers](https://huggingface.co/transformers/), a State-of-the-art Natural Language Processing library by [Hugging Face](https://huggingface.co/).\n\nBelow is the list of all the requirements that you might want to install through pip. It is recommended that you install all these packages inside a [virtual environment](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment) before proceeding.\n\n1. Jupyter Notebook\n2. Torch\n3. Librosa\n4. Numpy\n5. Soundfile\n6. Scipy\n7. IPython\n8. Transformers\n\n## Working with the code\n\nOpen your Jupyter notebook while activating the virtual environment that contains all the essential libraries mentioned above.\n\nOnce the notebook is loaded, create a new file and begin importing the libraries.\n\n\u003cpre\u003e\u003ccode\u003e\nimport torch\nimport librosa\nimport numpy as np\nimport soundfile as sf\nfrom scipy.io import wavfile\nfrom IPython.display import Audio\nfrom transformers import Wav2Vec2ForCTC, Wav2Vec2Tokenizer\n\u003c/code\u003e\u003c/pre\u003e\n\nNow that we have successfully imported all the libraries, let’s load our tokenizer and the model. We will be using a pre-trained model for this example. A pre-trained model is a model that has already been trained by someone else which we can reuse in our system. The model we are going to import is trained by Facebook.\n\n\u003cpre\u003e\u003ccode\u003e\ntokenizer = Wav2Vec2Tokenizer.from_pretrained(\"facebook/wav2vec2-base-960h\")\nmodel = Wav2Vec2ForCTC.from_pretrained(\"facebook/wav2vec2-base-960h\")\n\u003c/code\u003e\u003c/pre\u003e\n\nThis might take some time to download.\n\nOnce done, you can record your voice and save the wav file just next to the file you are writing your code in. You can name your audio to “my-audio.wav”.\n\n\u003cpre\u003e\u003ccode\u003e\nfile_name = 'my-audio.wav'\nAudio(file_name)\n\u003c/code\u003e\u003c/pre\u003e\n\nWith this code, you can play your audio in the Jupyter notebook.\n\nNext up: We will load our audio file and check our sample rate and total time.\n\n\u003cpre\u003e\u003ccode\u003e\ndata = wavfile.read(file_name)\nframerate = data[0]\nsounddata = data[1]\ntime = np.arange(0,len(sounddata))/framerate\nprint('Sampling rate:',framerate,'Hz')\n\u003c/code\u003e\u003c/pre\u003e\n\nThe result from this example would vary from yours as you may be using your own audio.\n\n\u003cpre\u003e\u003ccode\u003e\nSampling rate: 44100 Hz\n\u003c/code\u003e\u003c/pre\u003e\n\nThis is printed out for my sample audio.\n\nWe have to convert the sample rate to 16000 Hz as Facebook’s model accepts the sampling rate at this range. We will take help from Librosa, a library that we had installed earlier.\n\n\u003cpre\u003e\u003ccode\u003e\ninput_audio, _ = librosa.load(file_name, sr=16000)\n\u003c/code\u003e\u003c/pre\u003e\n\nFinally, the input audio is fed to the tokenizer which is then processed by the model. The final result will be stored in the transcription variable.\n\n\u003cpre\u003e\u003ccode\u003e\ninput_values = tokenizer(input_audio, return_tensors=\"pt\").input_values\nlogits = model(input_values).logits\npredicted_ids = torch.argmax(logits, dim=-1)\ntranscription = tokenizer.batch_decode(predicted_ids)[0]\nprint(transcription)\n\u003c/code\u003e\u003c/pre\u003e\n\nThe output:\n\n\u003cpre\u003e\u003ccode\u003e\n'DEEP LEARNING IS AMAZING'\n\u003c/code\u003e\u003c/pre\u003e\n\nWell, that is exactly what I had recorded in the my-audio.wav file.\n\nGo on and try recording your own speech and re-execute the last block of code. It should work within seconds.\n\n## [License](https://github.com/psavarmattas/SpeechToText/blob/master/LICENSE.MD)\n\nCopyright 2021 PSMForums\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsavarmattas%2Fspeechtotext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpsavarmattas%2Fspeechtotext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpsavarmattas%2Fspeechtotext/lists"}