{"id":16100790,"url":"https://github.com/pannous/caffe-speech-recognition","last_synced_at":"2025-10-10T09:52:11.351Z","repository":{"id":24637435,"uuid":"28046729","full_name":"pannous/caffe-speech-recognition","owner":"pannous","description":"Speech Recognition with the Caffe deep learning framework, migrating to ","archived":false,"fork":false,"pushed_at":"2018-11-20T12:44:05.000Z","size":65467,"stargazers_count":325,"open_issues_count":5,"forks_count":124,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-05-19T13:07:28.953Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/pannous/tensorflow-speech-recognition","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pannous.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-15T17:03:12.000Z","updated_at":"2024-08-02T09:57:49.000Z","dependencies_parsed_at":"2022-07-25T11:17:44.854Z","dependency_job_id":null,"html_url":"https://github.com/pannous/caffe-speech-recognition","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pannous/caffe-speech-recognition","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pannous%2Fcaffe-speech-recognition","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pannous%2Fcaffe-speech-recognition/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pannous%2Fcaffe-speech-recognition/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pannous%2Fcaffe-speech-recognition/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pannous","download_url":"https://codeload.github.com/pannous/caffe-speech-recognition/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pannous%2Fcaffe-speech-recognition/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279003414,"owners_count":26083582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T18:48:13.481Z","updated_at":"2025-10-10T09:52:11.334Z","avatar_url":"https://github.com/pannous.png","language":"Jupyter Notebook","funding_links":[],"categories":["\u003ca name=\"Speech\"\u003e4. Speech"],"sub_categories":[],"readme":"Speech Recognition with BVLC caffe\n==================================\n\nSpeech Recognition with the [caffe](https://github.com/BVLC/caffe) deep learning framework\n\nUPDATE: We are migrating to [tensorflow](https://github.com/pannous/tensorflow-speech-recognition/)\n\nThis project is quite fresh and only the first of three milestones is accomplished: \nEven now it might be useful if you just want to train a handful of commands/options (1,2,3..yes/no/cancel/...)\n\n1)  training spoken **numbers**:\n  * get spectogram training images from http://pannous.net/spoken_numbers.tar (470 MB)\n  * start ./train.sh\n  * test with `ipython notebook test-speech-recognition.ipynb`\n    or `caffe test ...` or `\u003ccaffe-root\u003e/python/classify.py`\n  * 99% accuracy, nice!\n  * online recognition and learning with `./recognition-server.py` and `./record.py` scripts\n\n![Sample spectrogram, That's what she said, too laid?](https://raw.githubusercontent.com/pannous/caffe-speech-recognition/master/0_Karen_160.png)\n\nSample spectrogram, Karen uttering 'zero' with 160 words per minute.\n\n\n2) training **words**:\n * 4GB of training [data](https://dl.dropboxusercontent.com/u/23615316/spoken_words.tar)\n * net topology: work in progress ...\n * todo: use [upcoming new](https://github.com/BVLC/caffe/issues/1653) caffe [LSTM](https://en.wikipedia.org/wiki/Long_short_term_memory) layers etc\n * UPDATE [LSTMs get rolling](https://github.com/BVLC/caffe/pull/1873), [still not merged](https://github.com/BVLC/caffe/pull/2033)\n * UPDATE since the caffe project leaders have a hindering merging policy and this pull request was shifted many times without ever being merged, we are migrating to [tensorflow](https://github.com/pannous/tensorflow-speech-recognition)\n * todo: add extra categories for a) silence b) common noises like typing, achoo c) ALL other noises\n\n\n3) training **speech**:\n * todo!\n * 100GB of training data here: http://www.openslr.org/12/\n * [TIMIT dataset](https://catalog.ldc.upenn.edu/memberships) $27,000.00 membership fee or [$250 for non-members](https://catalog.ldc.upenn.edu/LDC93S1)+[$2400 under research-only license](https://catalog.ldc.upenn.edu/LDC2016MNP)?\n * combine with google n-grams\n\n\nTheoretical background: **papers**\n\nA. Graves and N. Jaitly. Towards end-to-end speech recognition with recurrent neural networks. [In ICML, 2014](https://duckduckgo.com/l/?kh=-1\u0026uddg=http%3A%2F%2Fjmlr.org%2Fproceedings%2Fpapers%2Fv32%2Fgraves14.pdf)\n\nO. Vinyals, S. V. Ravuri, and D. Povey. Revisiting recurrent neural networks for robust ASR. [In ICASSP, 2012](http://research.microsoft.com/pubs/164627/4085.pdf)\n\n[Andrew Ng et al](http://arxiv.org/pdf/1406.7806.pdf) / [Baidu](http://arxiv.org/abs/1412.5567)\n\n[Hinton et al / Toronto](http://www.cs.toronto.edu/~hinton/absps/RNN13.pdf)\n\n[good old Hinton](http://psych.stanford.edu/~jlm/pdfs/Hinton12IEEE_SignalProcessingMagazine.pdf)\n\n[Schmidhuber et al](http://arxiv.org/pdf/1402.3511v1.pdf) using new 'ClockWork-RNNs'\n\nThe **book**:\n[Automatic Speech Recognition: A Deep Learning Approach](http://www.amazon.com/Automatic-Speech-Recognition-Communication-Technology/dp/1447157788/ref=sr_1_1?ie=UTF8\u0026qid=1422013427\u0026sr=8-1\u0026keywords=speech+recognition)  (Signals and Communication Technology) Hardcover – November 11, 2014 by Dong Yu (Author) and Li Deng (Author) \n\n\n**Related work**\n\nAlso see the [Kaldi](http://kaldi.sourceforge.net/about.html) project, which seems a bit messy but already uses deep learning with [LSTM](https://en.wikipedia.org/wiki/Long_short_term_memory)\nAnother experimental LSTM network, which works out-of-the-box: [Currennt](http://sourceforge.net/projects/currennt/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpannous%2Fcaffe-speech-recognition","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpannous%2Fcaffe-speech-recognition","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpannous%2Fcaffe-speech-recognition/lists"}